Skip to content
This repository was archived by the owner on Apr 25, 2024. It is now read-only.

Commit cca21f9

Browse files
issc29issc29
committed
Update Readme
1 parent d590fc7 commit cca21f9

File tree

2 files changed

+185
-159
lines changed

2 files changed

+185
-159
lines changed

‎.travis.yml‎

Lines changed: 0 additions & 3 deletions
This file was deleted.

‎README.md‎

Lines changed: 185 additions & 156 deletions
Original file line numberDiff line numberDiff line change
@@ -1,163 +1,192 @@
1-
# tar-fs
2-
3-
filesystem bindings for [tar-stream](https://github.com/mafintosh/tar-stream).
4-
5-
```
6-
npm install tar-fs
7-
```
8-
9-
[![build status](https://secure.travis-ci.org/mafintosh/tar-fs.png)](http://travis-ci.org/mafintosh/tar-fs)
10-
11-
## Usage
12-
13-
tar-fs allows you to pack directories into tarballs and extract tarballs into directories.
14-
15-
It doesn't gunzip for you, so if you want to extract a `.tar.gz` with this you'll need to use something like [gunzip-maybe](https://github.com/mafintosh/gunzip-maybe) in addition to this.
16-
17-
``` js
18-
var tar = require('tar-fs')
19-
var fs = require('fs')
20-
21-
// packing a directory
22-
tar.pack('./my-directory').pipe(fs.createWriteStream('my-tarball.tar'))
23-
24-
// extracting a directory
25-
fs.createReadStream('my-other-tarball.tar').pipe(tar.extract('./my-other-directory'))
26-
```
27-
28-
To ignore various files when packing or extracting add a ignore function to the options. `ignore`
29-
is also an alias for `filter`. Additionally you get `header` if you use ignore while extracting.
30-
That way you could also filter by metadata.
31-
32-
``` js
33-
var pack = tar.pack('./my-directory', {
34-
ignore: function(name) {
35-
return path.extname(name) === '.bin' // ignore .bin files when packing
36-
}
37-
})
38-
39-
var extract = tar.extract('./my-other-directory', {
40-
ignore: function(name) {
41-
return path.extname(name) === '.bin' // ignore .bin files inside the tarball when extracing
42-
}
43-
})
44-
45-
var extractFilesDirs = tar.extract('./my-other-other-directory', {
46-
ignore: function(_, header) {
47-
// pass files & directories, ignore e.g. symlinks
48-
return header.type !== 'file' && header.type !== 'directory'
49-
}
50-
})
51-
```
52-
53-
You can also specify which entries to pack using the `entries` option
54-
55-
```js
56-
var pack = tar.pack('./my-directory', {
57-
entries: ['file1', 'subdir/file2'] // only the specific entries will be packed
58-
})
59-
```
60-
61-
If you want to modify the headers when packing/extracting add a map function to the options
1+
# Code Scanning Javascript Tutorial
622

63-
``` js
64-
var pack = tar.pack('./my-directory', {
65-
map: function(header) {
66-
header.name = 'prefixed/'+header.name
67-
return header
68-
}
69-
})
3+
Welcome to the Code Scanning Javascript Tutorial! This tutorial will take you through how to set up Github Advanced Security: Code Scanning as well as interpret results that it may find. The following repository contains vulnerability [CVE-2018-20835](https://github.com/advisories/GHSA-x2mc-8fgj-3wmr) (aka Zip Slip) that was found by the [GitHub Security Lab](https://securitylab.github.com/).
704

71-
var extract = tar.extract('./my-directory', {
72-
map: function(header) {
73-
header.name = 'another-prefix/'+header.name
74-
return header
75-
}
76-
})
77-
```
5+
## Introduction
786

79-
Similarly you can use `mapStream` incase you wanna modify the input/output file streams
7+
Code scanning is a feature that you use to analyze the code in a GitHub repository to find security vulnerabilities and coding errors. Any problems identified by the analysis are shown in GitHub.
808

81-
``` js
82-
var pack = tar.pack('./my-directory', {
83-
mapStream: function(fileStream, header) {
84-
if (path.extname(header.name) === '.js') {
85-
return fileStream.pipe(someTransform)
86-
}
87-
return fileStream;
88-
}
89-
})
9+
You can use code scanning with CodeQL, a semantic code analysis engine. CodeQL treats code as data, allowing you to find potential vulnerabilities in your code with greater confidence than traditional static analyzers.
9010

91-
var extract = tar.extract('./my-directory', {
92-
mapStream: function(fileStream, header) {
93-
if (path.extname(header.name) === '.js') {
94-
return fileStream.pipe(someTransform)
95-
}
96-
return fileStream;
97-
}
98-
})
99-
```
11+
This tutorial with use CodeQL Analysis with Code Scanning in order to search for vulnerabilities within your code.
10012

101-
Set `options.fmode` and `options.dmode` to ensure that files/directories extracted have the corresponding modes
13+
## Instructions
10214

103-
``` js
104-
var extract = tar.extract('./my-directory', {
105-
dmode: parseInt(555, 8), // all dirs should be readable
106-
fmode: parseInt(444, 8) // all files should be readable
107-
})
108-
```
109-
110-
It can be useful to use `dmode` and `fmode` if you are packing/unpacking tarballs between *nix/windows to ensure that all files/directories unpacked are readable.
111-
112-
Alternatively you can set `options.readable` and/or `options.writable` to set the dmode and fmode to readable/writable.
113-
114-
``` js
115-
var extract = tar.extract('./my-directory', {
116-
readable: true, // all dirs and files should be readable
117-
writable: true, // all dirs and files should be writable
118-
})
119-
```
120-
121-
Set `options.strict` to `false` if you want to ignore errors due to unsupported entry types (like device files)
122-
123-
To dereference symlinks (pack the contents of the symlink instead of the link itself) set `options.dereference` to `true`.
124-
125-
## Copy a directory
126-
127-
Copying a directory with permissions and mtime intact is as simple as
128-
129-
``` js
130-
tar.pack('source-directory').pipe(tar.extract('dest-directory'))
131-
```
132-
133-
## Interaction with [`tar-stream`](https://github.com/mafintosh/tar-stream)
134-
135-
Use `finalize: false` and the `finish` hook to
136-
leave the pack stream open for further entries (see
137-
[`tar-stream#pack`](https://github.com/mafintosh/tar-stream#packing)),
138-
and use `pack` to pass an existing pack stream.
139-
140-
``` js
141-
var mypack = tar.pack('./my-directory', {
142-
finalize: false,
143-
finish: function(sameAsMypack) {
144-
mypack.entry({name: 'generated-file.txt'}, "hello")
145-
tar.pack('./other-directory', {
146-
pack: sameAsMypack
147-
})
148-
}
149-
})
150-
```
151-
152-
153-
## Performance
154-
155-
Packing and extracting a 6.1 GB with 2496 directories and 2398 files yields the following results on my Macbook Air.
156-
[See the benchmark here](https://gist.github.com/mafintosh/8102201)
157-
158-
* tar-fs: 34.261 ms
159-
* [node-tar](https://github.com/isaacs/node-tar): 366.123 ms (or 10x slower)
160-
161-
## License
162-
163-
MIT
15+
<details>
16+
<summary>Fork this repo</summary>
17+
<p>
18+
19+
Begin by [forking this repo](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/fork-a-repo).
20+
</p>
21+
</details>
22+
23+
<details>
24+
<summary>Enable Code Scanning</summary>
25+
<p>
26+
27+
#### Security tab
28+
29+
Click on the `Security` tab.
30+
31+
32+
<img src="https://user-images.githubusercontent.com/6920330/96745784-81480380-1394-11eb-886d-55e7c207c9c9.png" width="70%"/>
33+
34+
#### Set up code scanning
35+
36+
Click `Set up code scanning`.
37+
38+
<img src="https://user-images.githubusercontent.com/6920330/96745792-8311c700-1394-11eb-83fd-e47d09bf148e.png" width="70%"/>
39+
40+
#### Setup Workflow
41+
42+
Click the `Setup this workflow` button by CodeQL Analysis.
43+
44+
<img src="https://user-images.githubusercontent.com/6920330/96746928-aee17c80-1395-11eb-9eb2-657dd0e92ed9.png" width="70%"/>
45+
46+
This will create a GitHub Actions Workflow file with CodeQL already set up. Since Javascript is an interpreted language there is no need to configure any builds. See the [documentation](https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/running-codeql-code-scanning-in-your-ci-system) if you would like to configure CodeQL Analysis with a 3rd party CI system instead of using GitHub Actions.
47+
</p>
48+
</details>
49+
50+
<details>
51+
52+
<summary>Actions Workflow file</summary>
53+
<p>
54+
55+
#### Actions Workflow
56+
57+
The Actions Workflow file contains a number of different sections including:
58+
1. Checking out the repository
59+
2. Initializing the CodeQL Action
60+
3. Running Autobuilder (not necessary for interpreted languages)
61+
4. Running the CodeQL Analysis
62+
63+
<img src="https://user-images.githubusercontent.com/6920330/96746940-b143d680-1395-11eb-9778-0891525a39c1.png" width="80%"/>
64+
65+
Click `Start Commit` -> `Commit this file` to commit the changes.
66+
</p>
67+
</details>
68+
69+
<details>
70+
<summary>GitHub Actions Progress</summary>
71+
72+
<p>
73+
74+
#### GitHub Actions Progress
75+
76+
Click `Actions` tab -> `CodeQL`
77+
78+
Click the specific workflow run. You can view the progress of the Workflow run until the analysis completes.
79+
80+
<img src="https://user-images.githubusercontent.com/6920330/96748337-64f99600-1397-11eb-9ab7-b78ec23466ae.png" width="80%"/>
81+
82+
</p>
83+
</details>
84+
85+
<details>
86+
<summary>Security Issues</summary>
87+
<p>
88+
89+
Once the Workflow has completed, click the `Security` tab -> ` Code Scanning Alerts`. An security alert "Arbitrary file write during zip extraction ("Zip Slip")
90+
" should be visible.
91+
92+
#### Security Alert View
93+
94+
Clicking on the security alert will provide details about the security alert including:
95+
A description of the issue
96+
A tag to the CWE that it is connected to as well as the type of alert (Error, Warning, Note)
97+
The line of code that triggered the security alert
98+
The ability to dismiss the alert depending on certain conditions (false positive? won't fix? used in tests?)
99+
100+
<img src="https://user-images.githubusercontent.com/6920330/96749627-0c2afd00-1399-11eb-92f9-3356e387201f.png" width="80%"/>
101+
102+
#### Security Alert Description
103+
104+
Click `Show more` to view a full desciption of the alert including examples and links to additional information.
105+
106+
<img src="https://user-images.githubusercontent.com/6920330/96749626-0c2afd00-1399-11eb-9ef5-9560ca585a1b.png" width="80%"/>
107+
108+
#### Security Full Description
109+
110+
<img width="80%" src="https://user-images.githubusercontent.com/6920330/97208478-8bea0a80-1791-11eb-8a2a-f625649312f0.png">
111+
112+
</p>
113+
</details>
114+
115+
<details>
116+
<summary>Show Paths</summary>
117+
<p>
118+
119+
#### Show Paths Button
120+
121+
CodeQL Analysis is able to trace the dataflow path from source to sink and gives you the ability to view the path traversal within the alert.
122+
123+
Click `show paths` in order to see the dataflow path that resulted in this alert.
124+
125+
<img src="https://user-images.githubusercontent.com/6920330/96749839-514f2f00-1399-11eb-80f7-1b83e5c195e7.png" width="80%"/>
126+
127+
#### Show Paths View
128+
129+
<img src="https://user-images.githubusercontent.com/6920330/96749909-6926b300-1399-11eb-99df-143d17804aeb.png" width="80%"/>
130+
131+
</p>
132+
</details>
133+
134+
<details>
135+
<p>
136+
137+
<summary>Fix the Security Alert</summary>
138+
139+
In order to fix this specific alert, we will need to ensure that the destination file paths is the only location where files can be written to.
140+
141+
Click on the `Code` tab and [Edit](https://docs.github.com/en/free-pro-team@latest/github/managing-files-in-a-repository/editing-files-in-your-repository) the `index.js` file. Navigate to Line 264 of the `index.js` file and modify the line:
142+
143+
`var srcpath = path.resolve(cwd, header.linkname)`
144+
145+
to
146+
147+
`var srcpath = path.join(cwd, path.join('/', header.linkname))`
148+
149+
Click `Create a new branch for this commit and start a pull request`, name the branch `fix-zip-slip`, and create the Pull Request.
150+
151+
#### Pull Request Status Check
152+
153+
In the Pull Request, you will notice that the CodeQL Analysis has started as a status check. Wait until it completes.
154+
155+
<img src="https://user-images.githubusercontent.com/6920330/96752215-2adec300-139c-11eb-9c5e-3a04f24ba0bf.png" width="80%"/>
156+
157+
#### Security Alert Details
158+
159+
After the Workflow has completed click on `Details` by the `Code Scanning Results / CodeQL` status check.
160+
161+
<img src="https://user-images.githubusercontent.com/6920330/96752487-85781f00-139c-11eb-943d-602f2de98998.png" width="80%"/>
162+
163+
#### Fixed Alert
164+
165+
Notice that Code Scanning has detected that this Pull Request will fix the Zip Slip vulnerability that was detected before.
166+
167+
<img src="https://user-images.githubusercontent.com/6920330/96752486-85781f00-139c-11eb-9a7e-3ccbc81da3d1.png" width="80%"/>
168+
169+
Merge the Pull Request. After the Pull Request has been merged, another Workflow will kick off to scan the repository for any vulnerabilties.
170+
171+
#### Closed Security Alerts
172+
173+
After the final Workflow has completed, navigate back to the `Security` tab and click `Closed`. Notice that the Zip Slip security alert now shows up as a closed issue.
174+
175+
<img src="https://user-images.githubusercontent.com/6920330/96753441-e0f6dc80-139d-11eb-9a2a-d53075b6331e.png" width="80%"/>
176+
177+
#### Traceability
178+
179+
Click on the security alert and notice that it details when the fix was made, by whom, and the specific commit. This provides full traceability to detail when and how a security alert was fixed and exactly what was changed to remediate the issue.
180+
181+
<img src="https://user-images.githubusercontent.com/6920330/96753440-e05e4600-139d-11eb-81ed-c22e4f41d74a.png" width="80%"/>
182+
183+
</p>
184+
</details>
185+
186+
## Next Steps
187+
188+
Ready to talk about advanced security features for GitHub Enterprise? [Contact Sales](https://enterprise.github.com/contact) for more information!
189+
190+
Check out [GitHub's Security feature page](https://github.com/features/security) for more security features embedded into GitHub.
191+
192+
Check out the Code Scanning [documentation](https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/about-code-scanning) for additional configuration options and technical details.

0 commit comments

Comments
 (0)