2570 scan metrics #906
Draft
2570 scan metrics #906
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ch scan/organization pair.
Add query to list_scans to count total organizations and organizations that were successfully scanned by each scan. Add total_orgs and orgs_with_results fields to scan models and interfaces. Add optional window_days argument to /scans endpoint to specify window for orgs_with_results query. Update fetchScans call in ScansView.tsx to include window_days.
Add Success Rate column and associated tooltips to Scans page. Define windowDays state for get /scans endpoint and tooltips. Increase width of Admin Tools container to better display tables.
… total_orgs; improve logging for launch_scan_execution.
…an Organization Rename update_scan_result helper function to upsert_scan_result Add logic to determine if a scan returns and saves a result for censys, censys_sync, shodan, and shodan_sync Add logic to upsert timestamp to scan_result table when a scan returns and saves a result for censys, censys_sync, shodan, and shodan_sync
Add logic to dns_twist.py to update scan_result table if domains are found and saved Add logic to intel_x_identity.py to update scan_result table if both breaches and credentials are found and saved Move upsert_scan_result.py from xfd_api.helpers to xfd_api.tasks.helpers
…h organization scanned by cybersixgill; add self.scan_id to intel_x __init__.
…sync. Add logic to credential_sync and sync_asm_sync for recording timestamp of last successful scan per organization. Add 204 status code to dns_twist for identifying instances where no results are saved.
Refactor Logger.error to use lazy % formatting in censys_sync.py. Extract queue creation logic from launch_scan_execution to address too_many_statements issue.
Add window_days argument Add annotation to calculate orgs_with_results Add error handling to match list_scans method
Add total_orgs and orgs_with_results to scan model in list_scan_tasks. Extract scan query logic to helper function, query_scans, for use by scan and scan_task api methods.
Fix order for arguments to address lint issues Consolidate default value for window_days by removing it from ScansView.tsx
asm_sync.py: - Improve error handling and logging - Add flags to monitor success rate of retrieving both CIDRs and Subdomains - Track success rate using upsert_scan_result helper function - Delete commented-out code credential_sync.py: - Fix logic for updating data_saved - Revert code to return status 200 instead of 204 if no data is updated scan.py - Modify list_scans and get_scan to return metrics_window_days shodan.py: - Improve logging
Update logging for asm_sync Fix argument order for upsert_scan_results in cybersixgill, dnstwist, and intelx Update default backend domain for local blocklist queries in dnstwist Replace f strings in dnstwist Replace print statements with logging statements in upsert_scan_result helper
Add metrics window to scan response model Display metrics window in success rate tooltips in scans view Add tooltip to success rate header in scans view Removes unused imports and commented out code from scans view
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗣 Description
💭 Motivation and context
🧪 Testing
✅ Pre-approval checklist
in code comments.
to reflect the changes in this PR.
✅ Pre-merge checklist
✅ Post-merge checklist