You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/contributing/how_to/testing.mdx
+41Lines changed: 41 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -124,6 +124,47 @@ start "" htmlcov/index.html || open htmlcov/index.html
124
124
125
125
```
126
126
127
+
## Snapshot Testing
128
+
129
+
Some tests use [syrupy](https://github.com/tophat/syrupy) for snapshot testing, which captures the output of functions and compares them to stored snapshots. This is particularly useful for testing JSON schema generation and other structured outputs.
130
+
131
+
### Updating Snapshots
132
+
133
+
To update snapshots when the expected output has legitimately changed:
134
+
135
+
```bash
136
+
uv run --group test pytest path/to/test.py --snapshot-update
137
+
```
138
+
139
+
### Pydantic Version Compatibility Issues
140
+
141
+
Pydantic generates different JSON schemas across versions, which can cause snapshot test failures in CI when tests run with different Pydantic versions than what was used to generate the snapshots.
142
+
143
+
**Symptoms:**
144
+
- CI fails with snapshot mismatches showing differences like missing or extra fields.
145
+
- Tests pass locally but fail in CI with different Pydantic versions
146
+
147
+
**Solution:**
148
+
Locally update snapshots using the same Pydantic version that CI uses:
149
+
150
+
1.**Identify the failing Pydantic version** from CI logs (e.g., `2.7.0`, `2.8.0`, `2.9.0`)
151
+
152
+
2.**Update snapshots with that version:**
153
+
```bash
154
+
uv run --with "pydantic==2.9.0" --group test pytest tests/unit_tests/path/to/test.py::test_name --snapshot-update
155
+
```
156
+
157
+
3.**Verify compatibility across supported versions:**
158
+
```bash
159
+
# Test with the version you used to update
160
+
uv run --with "pydantic==2.9.0" --group test pytest tests/unit_tests/path/to/test.py::test_name
161
+
162
+
# Test with other supported versions
163
+
uv run --with "pydantic==2.8.0" --group test pytest tests/unit_tests/path/to/test.py::test_name
164
+
```
165
+
166
+
**Note:** Some tests use `@pytest.mark.skipif` decorators to only run with specific Pydantic version ranges (e.g., `PYDANTIC_VERSION_AT_LEAST_210`). Make sure to understand these constraints when updating snapshots.
167
+
127
168
## Coverage
128
169
129
170
Code coverage (i.e. the amount of code that is covered by unit tests) helps identify areas of the code that are potentially more or less brittle.
0 commit comments