Reporting rework#24

Draft

rkoumis wants to merge 138 commits intomasterfrom

reporting-rework

Commits on Feb 5, 2025

Refactoring of mongodb constants
rkoumis
committed
Tests for dev_utils/mongodb.py
rkoumis
committed
Add mongomock for testing MongoDB code
rkoumis
committed
More tests with mongomock
rkoumis
committed
add reporting to core

Nick Bargnesi
authored and
rkoumis
committed
swap direct to MongoDB/ES for core.reporting

Nick Bargnesi
authored and
rkoumis
committed
implement some APIs in the mongodb backend

Nick Bargnesi
authored and
rkoumis
committed
Initial schema for summary
rkoumis
committed
remove post_processing v2 API

Nick Bargnesi
authored and
rkoumis
committed
remove Shrike support

Nick Bargnesi
authored and
rkoumis
committed
add basic tests, use getattribute vice getattr

Nick Bargnesi
authored and
rkoumis
committed
use old Black line lengths with Ruff

Nick Bargnesi
authored and
rkoumis
committed
add a TODO ("delete_mongo")

Nick Bargnesi
authored and
rkoumis
committed
Ruff is happy now 🐶

Nick Bargnesi
authored and
rkoumis
committed
add a reporting behavior API

Nick Bargnesi
authored and
rkoumis
committed
add behavior backend impls

Nick Bargnesi
authored and
rkoumis
committed
start using the reporting behavior API

Nick Bargnesi
authored and
rkoumis
committed
Added draft implementations and stub schema
rkoumis
committed
Get tests running again
rkoumis
committed
Address import error from elasticsearch
rkoumis
committed
Update test_web_utils.py
rkoumis
committed
Added two basic tests for MongoDBReports
rkoumis
committed
Added info schema nested in summary
rkoumis
committed
add API dropped/memory/network/procdump/procmemory

Nick Bargnesi
authored and
rkoumis
committed
simplify DISABLED_WEB using reporting.enabled

Nick Bargnesi
authored and
rkoumis
committed
use reporting API in the ajax load_files handler

Nick Bargnesi
authored and
rkoumis
committed
Add more schema objects
rkoumis
committed
Add pydantic to poetry dependencies

josh-feather
authored and
rkoumis
committed
Move BaseModel to Pydantic 2 syntax

josh-feather
authored and
rkoumis
committed
Add Behaviour-related Pydantic Models

josh-feather
authored and
rkoumis
committed
Change to using python 3.10 syntax for optional
rkoumis
committed
Added default value of None for optional
rkoumis
committed
Yes we can have nested model classes
rkoumis
committed
Use from future annotations to postpone evaluation
rkoumis
committed
Added API Call type to schema
rkoumis
committed
add/change TODOs

Nick Bargnesi
authored and
rkoumis
committed
rewrite fetch_signature_call_data; mark TODO

Nick Bargnesi
authored and
rkoumis
committed
rewrite signature_calls; can use the existing API

Nick Bargnesi
authored and
rkoumis
committed
rewrite filtered_chunk; needs PID lookup of calls

Nick Bargnesi
authored and
rkoumis
committed
rewrite antivirus; just use reporting API get

Nick Bargnesi
authored and
rkoumis
committed
rewrite Suricata handlers

Nick Bargnesi
authored and
rkoumis
committed
rewrite search behavior

Nick Bargnesi
authored and
rkoumis
committed
rewrite report function; go right to reports.get

Nick Bargnesi
authored and
rkoumis
committed
rewrite and annotate report memory

Nick Bargnesi
authored and
rkoumis
committed
swap out enabledconf checks for reporting.enabled

Nick Bargnesi
authored and
rkoumis
committed
rewrite procdump to use reports.procmemory

Nick Bargnesi
authored and
rkoumis
committed
start using reporting APIs in search

Nick Bargnesi
authored and
rkoumis
committed
use reporting APIs in remove handler, annotate bug

Nick Bargnesi
authored and
rkoumis
committed
use reporting API in pcapstream

Nick Bargnesi
authored and
rkoumis
committed
start implementing comments using reporting API

Nick Bargnesi
authored and
rkoumis
committed
pull unused imports and format web analysis views

Nick Bargnesi
authored and
rkoumis
committed
Remove Behavior.ProcessCall; ref Call instead

josh-feather
authored and
rkoumis
committed
Use a fork of pyattck during CI
rkoumis
committed
Remove unused import, fix behavior test
rkoumis
committed
use dot imports accessing relative tests modules
Nick Bargnesi
committed
regen poetry.lock
Nick Bargnesi
committed
Revert "use dot imports accessing relative tests modules"
josh-feather
committed
Set import mode in config
rkoumis
committed
make python-magic a real dependency
Nick Bargnesi
committed
Tentative schema for proc-memory
rkoumis
committed
Use the CAPESandbox fork of pyattck
rkoumis
committed

Commits on Feb 6, 2025

Commits on Feb 7, 2025

rewrite admin.remove to use the reporting API
Nick Bargnesi
committed
replace db direct access to MongoDB/Elastic
Nick Bargnesi
committed
add analyses and task path conveniences
Nick Bargnesi
committed
use get_task_path in database
Nick Bargnesi
committed
Fix reference error
josh-feather
committed
Simplify Calls APIs
josh-feather
committed
Add new API signatures
josh-feather
committed
Change to
josh-feather
committed
Remove duplicate test
josh-feather
committed
Make reporting API search cputime unlimited
josh-feather
committed
Update return type for
josh-feather
committed
Formatting
josh-feather
committed
Remove duplicate `get_functions` func & remove unused imports in backend tests
josh-feather
committed
Implement search_by_sha256 Mongo API
josh-feather
committed
replace MongoDB/Elastic in startup with reporting
Nick Bargnesi
committed
change return type of suricata methods
josh-feather
committed
Add limit params to all by_sha256 api methods
josh-feather
committed
Implement suricata reporting API methods
josh-feather
committed
Improve search_by_sha256 result handling
josh-feather
committed
Add backend responsive func
josh-feather
committed
Added default factory to two Fields
rkoumis
committed
Flesh out the Network schema object
rkoumis
committed
Added HTTP details to the Network schema
rkoumis
committed
Move the Machine class into the Info class
rkoumis
committed
Updated FileActivities in Behavior
rkoumis
committed
Minor tweaks
rkoumis
committed
rename check_webgui_mongo → check_reporting
Nick Bargnesi
committed
mark init_mongo for removal
Nick Bargnesi
committed
remove mongo_delete_data
Nick Bargnesi
committed
add a TODO for @josh-feather
Nick Bargnesi
committed
add TODOs for @josh-feather
Nick Bargnesi
committed
replace MongoDB/Elastic in cape_utils w/ reporting
Nick Bargnesi
committed
replace MongoDB/Elastic in cleaners with reporting
Nick Bargnesi
committed

Commits on Feb 10, 2025

Commits on Feb 12, 2025