Skip to content

Commit 4c9e026

Browse files
author
Sebastian Bauersfeld
committed
update
1 parent 163dc40 commit 4c9e026

28 files changed

+1527
-7
lines changed

‎.github/workflows/action-test.yml

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
name: "Action Test"
2+
on:
3+
workflow_dispatch:
4+
inputs:
5+
pattern1:
6+
description: 'Include or exclude pattern'
7+
required: true
8+
pattern2:
9+
description: 'Include or exclude pattern'
10+
required: false
11+
pattern3:
12+
description: 'Include or exclude pattern'
13+
required: false
14+
pattern4:
15+
description: 'Include or exclude pattern'
16+
required: false
17+
18+
jobs:
19+
analyze:
20+
name: Analyze
21+
runs-on: ubuntu-latest
22+
23+
strategy:
24+
fail-fast: false
25+
matrix:
26+
language: [ 'java' ]
27+
28+
steps:
29+
- name: Checkout repository
30+
uses: actions/checkout@v2
31+
32+
- name: Initialize CodeQL
33+
uses: github/codeql-action/init@v1
34+
with:
35+
languages: ${{ matrix.language }}
36+
37+
- run: |
38+
javatest/build
39+
40+
- name: Perform CodeQL Analysis
41+
uses: github/codeql-action/analyze@v1
42+
with:
43+
upload: False
44+
output: sarif-results
45+
46+
- name: filter-sarif
47+
uses: zbazztian/filter-sarif@master
48+
with:
49+
#patterns: |
50+
# -**/*.java
51+
# +**/*Exposed*
52+
patterns: |
53+
${{ github.event.inputs.pattern1 }}
54+
${{ github.event.inputs.pattern2 }}
55+
${{ github.event.inputs.pattern3 }}
56+
${{ github.event.inputs.pattern4 }}
57+
input: sarif-results/java-builtin.sarif
58+
output: sarif-results/java-builtin.sarif
59+
60+
- name: Upload SARIF
61+
uses: github/codeql-action/upload-sarif@v1
62+
with:
63+
sarif_file: sarif-results/java-builtin.sarif
64+
65+
- name: Upload loc as a Build Artifact
66+
uses: actions/[email protected]
67+
with:
68+
name: sarif-results
69+
path: sarif-results
70+
retention-days: 1

‎Dockerfile

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
FROM alpine:latest
2-
RUN apk --no-cache add python3
1+
FROM ubuntu:latest
2+
RUN apt-get update && apt-get install -y python3
33
COPY entrypoint.sh /entrypoint.sh
4+
COPY filter-sarif *.py /
45
ENTRYPOINT ["/entrypoint.sh"]

‎README.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# filter-sarif
2+
3+
Takes a SARIF file and a list of inclusion and exclusion patterns as input and removes alerts from the SARIF file according to those patterns.
4+
5+
# Example
6+
7+
The following example removes all alerts from all Java test files:
8+
9+
```yaml
10+
name: "Filter SARIF"
11+
on:
12+
push:
13+
branches: [master]
14+
15+
jobs:
16+
analyze:
17+
name: Analyze
18+
runs-on: ubuntu-latest
19+
20+
strategy:
21+
fail-fast: false
22+
matrix:
23+
language: [ 'java' ]
24+
25+
steps:
26+
- name: Checkout repository
27+
uses: actions/checkout@v2
28+
29+
- name: Initialize CodeQL
30+
uses: github/codeql-action/init@v1
31+
with:
32+
languages: ${{ matrix.language }}
33+
34+
- name: Autobuild
35+
uses: github/codeql-action/autobuild@v1
36+
37+
- name: Perform CodeQL Analysis
38+
uses: github/codeql-action/analyze@v1
39+
with:
40+
upload: False
41+
output: sarif-results
42+
43+
- name: filter-sarif
44+
uses: zbazztian/filter-sarif@master
45+
with:
46+
patterns: |
47+
+**/*.java
48+
-**/*Test*.java
49+
input: sarif-results/java-builtin.sarif
50+
output: sarif-results/java-builtin.sarif
51+
52+
- name: Upload SARIF
53+
uses: github/codeql-action/upload-sarif@v1
54+
with:
55+
sarif_file: sarif-results/java-builtin.sarif
56+
57+
- name: Upload loc as a Build Artifact
58+
uses: actions/[email protected]
59+
with:
60+
name: sarif-results
61+
path: sarif-results
62+
retention-days: 1
63+
```
64+
65+
Note how we provided `upload: False` and `output: sarif-results` to the `analyze` action. That way we can filter the SARIF with the `filter-sarif` action before uploading it via `upload-sarif`. Finally, we also attach the resulting SARIF file to the build, which is convenient for later inspection.
66+
67+
# Patterns
68+
69+
Each pattern line is of the form:
70+
```
71+
[+/-]<file pattern>[:<rule pattern>]
72+
```
73+
74+
for example:
75+
```
76+
-**/*Test*.java:** # exclusion pattern: remove all alerts from all Java test files
77+
-**/*Test*.java # ditto, short form of the line above
78+
+**/*.java:java/sql-injection # inclusion pattern: This line has precedence over the first two
79+
# and thus "whitelists" alerts of type "java/sql-injection"
80+
**/*.java:java/sql-injection # ditto, the "+" in inclusion patterns is optional
81+
** # allow all alerts in all files (reverses all previous lines)
82+
```
83+
84+
* The path separator character in patterns is always `/`, independent of the platform the code is running on and independent of the paths in the SARIF file.
85+
* `*` matches any character, except a path separator
86+
* `**` matches any character and is only allowed between path separators, e.g. `/**/file.txt`, `**/file.txt` or `**`. NOT allowed: `**.txt`, `/etc**`
87+
* The rule pattern is optional. If omitted, it will apply to alerts of all types.
88+
* Subsequent lines override earlier ones. By default all alerts are included.
89+
* If you need to use the literals `+`, `-`, `\` or `:` in your pattern, you can escape them with `\`, e.g. `\-this/is/an/inclusion/file/pattern\:with-a-semicolon:and/a/rule/pattern/with/a/\\/backslash`. For `+` and `-`, this is only necessary if they appear at the beginning of the pattern line.

‎action.yml

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,14 @@
1-
name: 'Filter Code Scanning paths'
2-
description: 'Filter Code Scanning paths'
1+
name: 'Filter SARIF'
2+
description: 'Filter SARIF results by path'
33
inputs:
4-
dir:
5-
description: 'directory to remove'
4+
patterns:
5+
description: 'Patterns for the results to remove. If a pattern starts with "-", it is an exclusion pattern, otherwise, it is an inclusion pattern.'
6+
required: true
7+
input:
8+
description: 'Path to the input SARIF file'
9+
required: true
10+
output:
11+
description: 'Path to the input SARIF file'
612
required: true
713
runs:
814
using: 'docker'

‎entrypoint.sh

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,8 @@
11
#!/bin/sh
2-
ls -lah
2+
unset LD_PRELOAD
3+
/filter-sarif \
4+
--input "/github/workspace/${INPUT_INPUT}" \
5+
--output "/github/workspace/${INPUT_OUTPUT}" \
6+
--split-lines \
7+
-- \
8+
"$INPUT_PATTERNS"

‎filter-sarif

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/bin/sh
2+
HERE="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
3+
python3 "${HERE}/filter_sarif.py" "$@"

‎filter_sarif.py

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
import sys
2+
import argparse
3+
import json
4+
import re
5+
from globber import match
6+
7+
8+
def fail(msg):
9+
print(msg)
10+
sys.exit(-1)
11+
12+
13+
def match_path_and_rule(path, rule, patterns):
14+
result = True
15+
for s, fp, rp in patterns:
16+
if match(rp, rule) and match(fp, path):
17+
result = s
18+
return result
19+
20+
21+
def parse_pattern(line):
22+
sepchar = ':'
23+
escchar = '\\'
24+
file_pattern = ''
25+
rule_pattern = ''
26+
seen_separator = False
27+
sign = True
28+
29+
# inclusion or exclusion pattern?
30+
uline = line
31+
if line:
32+
if line[0] == '-':
33+
sign = False
34+
uline = line[1:]
35+
elif line[0] == '+':
36+
uline = line[1:]
37+
38+
i = 0
39+
while i < len(uline):
40+
c = uline[i]
41+
i = i + 1
42+
if c == sepchar:
43+
if seen_separator:
44+
raise Exception('Invalid pattern: "' + line + '" Contains more than one separator!')
45+
seen_separator = True
46+
continue
47+
elif c == escchar:
48+
nextc = uline[i] if (i < len(uline)) else None
49+
if nextc in ['+' , '-', escchar, sepchar]:
50+
i = i + 1
51+
c = nextc
52+
if seen_separator:
53+
rule_pattern = rule_pattern + c
54+
else:
55+
file_pattern = file_pattern + c
56+
57+
if not rule_pattern:
58+
rule_pattern = '**'
59+
60+
return sign, file_pattern, rule_pattern
61+
62+
63+
def filter_sarif(args):
64+
if args.split_lines:
65+
tmp = []
66+
for p in args.patterns:
67+
tmp = tmp + re.split('\r?\n', p)
68+
args.patterns = tmp
69+
70+
args.patterns = [parse_pattern(p) for p in args.patterns if p]
71+
72+
print('Given patterns:')
73+
for s, fp, rp in args.patterns:
74+
print(
75+
'files: {file_pattern} rules: {rule_pattern} ({sign})'.format(
76+
file_pattern=fp,
77+
rule_pattern=rp,
78+
sign='positive' if s else 'negative'
79+
)
80+
)
81+
82+
with open(args.input, 'r') as f:
83+
s = json.load(f)
84+
85+
for run in s.get('runs', []):
86+
if run.get('results', []):
87+
new_results = []
88+
for r in run['results']:
89+
if r.get('locations', []):
90+
new_locations = []
91+
for l in r['locations']:
92+
# TODO: The uri field is optional. We might have to fetch the actual uri from "artifacts" via "index"
93+
# (see https://github.com/microsoft/sarif-tutorials/blob/main/docs/2-Basics.md#-linking-results-to-artifacts)
94+
uri = l.get('physicalLocation', {}).get('artifactLocation', {}).get('uri', None)
95+
# TODO: The ruleId field is optional and potentially ambiguous. We might have to fetch the actual
96+
# ruleId from the rule metadata via the ruleIndex field.
97+
# (see https://github.com/microsoft/sarif-tutorials/blob/main/docs/2-Basics.md#rule-metadata)
98+
ruleId = r['ruleId']
99+
if uri is None or match_path_and_rule(uri, ruleId, args.patterns):
100+
new_locations.append(l)
101+
r['locations'] = new_locations
102+
if new_locations:
103+
new_results.append(r)
104+
else:
105+
# locations array doesn't exist or is empty, so we can't match on anything
106+
# therefore, we include the result in the output
107+
new_results.append(r)
108+
run['results'] = new_results
109+
110+
with open(args.output, 'w') as f:
111+
json.dump(s, f, indent=2)
112+
113+
114+
def main():
115+
parser = argparse.ArgumentParser(
116+
prog='filter-sarif'
117+
)
118+
parser.add_argument(
119+
'--input',
120+
help='Input SARIF file',
121+
required=True
122+
)
123+
parser.add_argument(
124+
'--output',
125+
help='Output SARIF file',
126+
required=True
127+
)
128+
parser.add_argument(
129+
'--split-lines',
130+
default=False,
131+
action='store_true',
132+
help='Split given patterns on newlines.'
133+
)
134+
parser.add_argument(
135+
'patterns',
136+
help='Inclusion and exclusion patterns.',
137+
nargs='+'
138+
)
139+
140+
def print_usage(args):
141+
print(parser.format_usage())
142+
143+
args = parser.parse_args()
144+
filter_sarif(args)
145+
146+
147+
main()

0 commit comments

Comments
 (0)