wiki: ingest JobFilter product knowledge base - #31
Open
manazoid4 wants to merge 304 commits into
Open
Conversation
Two badges showed broken/stale state on the AI-Marketing-Hub private mirror: - `public release` resolved to v1.6.0 because that is the AgriciDaniel public-repo state. Truthful for the public, misleading on the private mirror which is at v1.9.0. - `public CI` returned shields.io "repo or workflow not found" because test.yml only exists on the private mirror; the public canonical has no workflow file yet. Swap both for static badges anchored to private-repo URLs: - release: static v1.9.0 -> private releases/latest - CI: static passing -> private actions (verified: 3/3 recent runs green) Update the footnote to make the dual-repo badge strategy explicit: star count tracks public (shields.io constraint); release + CI are pinned static for the private mirror until v1.7+ work promotes to the public canonical.
Three changes to improve readability and remove dead links for anyone who
is not an AI Marketing Hub Pro member:
1. Two-versions callout: replace the technical `AI-Marketing-Hub/claude-obsidian`
repo path (which 404s for non-members) with a link to the public org page
github.com/AI-Marketing-Hub. The org page is publicly reachable and lets
prospective members see the org profile + community link. Also rephrase
both bullets in parallel ("Public open-source build" / "Community
early-access build") and drop the path-style label in favor of a human
anchor ("Daniel Agrici's GitHub" / "AI Marketing Hub").
2. Quick Start opening note: insert a one-line disclaimer BEFORE Option 1
so public users see the URL swap from the start, not just under Option 2.
Pro members can ignore the note and use the commands as-is.
3. Related Projects: fix the claude-ads link to point at the public
github.com/AgriciDaniel/claude-ads mirror (which exists) instead of the
private AI-Marketing-Hub/claude-ads (which 404s for non-members).
scripts/contextual-prefix.py:376 used a direct write_text() to the chunk JSON file. A crash, SIGKILL, or ENOSPC mid-write left a half-written chunk-NNN.json which discover_chunks() then silently skipped via the JSONDecodeError catch at line 134. Result: silent loss of that chunk from BM25 + embed index until the page was re-ingested. Switch to tmp+rename using os.replace(), matching the pattern already established in scripts/bm25-index.py:178-186 for write_index(). The chunk JSON is now atomic at the filesystem boundary; crashes leave either the old chunk intact or the new chunk fully written. Verification: kill -9 of an in-flight contextual-prefix --build now leaves zero corrupted JSON files in .vault-meta/chunks/. Findings closed: B2 from docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md.
…dius (B1+H1)
The PostToolUse auto-commit hook used:
git add wiki/ .raw/ .vault-meta/
git diff --cached --quiet
git commit -m "wiki: auto-commit ..."
When the user had something else manually staged (TEST.md), the diff-cached
check returned non-empty and the commit swept TEST.md into the "wiki:
auto-commit" message. User intent buried under generic commit. The bug
self-demonstrated during this very fix cycle: the prior commit
(workspace-visual.json untracking, ought to have stayed staged for an
explicit chore commit) was swept up by the OLD hook under a wiki:
auto-commit message before this fix landed.
New hook command:
git add -- wiki/ .raw/ .vault-meta/
git diff --cached --quiet -- wiki/ .raw/ .vault-meta/
git commit -m "..." -- wiki/ .raw/ .vault-meta/
Three scoping changes:
1. add: explicit -- separator (H1, defends against a future wiki/-flag.md
filename being interpreted as a flag).
2. diff-cached: scoped pathspec, so the check only fires when wiki/-class
paths have staged changes.
3. commit: explicit -- pathspec, so only wiki/-class staged changes are
committed; anything else the user staged stays staged for their own
commit.
H3 (logged-error retry/surface) was previously addressed in v1.7.1 via
the .vault-meta/hook.log line that survives this change. The audit's
recommended counter + wiki-lint surface is deferred to v1.9.1.
Verification: python3 -c "import json; json.load(open('hooks/hooks.json'))"
parses OK; grep confirms the 3 critical pathspec changes.
Findings closed: B1 + H1 from docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md.
The 548d294 commit fixed README's two-versions callout, Quick Start disclaimer, and Related Projects claude-ads link to be reachable by public viewers (whose private repo URLs 404). The fix did NOT cascade to other user-facing surfaces. This commit completes the cascade. Files updated: - docs/install-guide.md: version 1.6.0 -> 1.9.0; top-line public canonical + early-access mirror split; new line-6 disclaimer mirroring README's Quick Start pattern; Support section bullets repointed to public. - CONTRIBUTING.md: clone command repointed to user fork of public canonical; Pro-member backport note added. - AGENTS.md: plugin homepage split into public canonical + community early-access mirror. - GEMINI.md: same split as AGENTS.md. - ATTRIBUTION.md: Repository field split into public canonical + community early-access mirror. - .claude-plugin/plugin.json: homepage + repository repointed to public canonical (these surface in Claude plugin UI; public visitors must click through, not 404). - .claude-plugin/marketplace.json: plugins[0].homepage + repository same swap as plugin.json. The marketplace catalog slug name (line 2, ai-marketing-hub-claude-obsidian) is the Pro marketplace identifier and stays. - docs/releases/v1.6.0.md: 3 raw.githubusercontent.com image URLs swapped from AI-Marketing-Hub/claude-obsidian to AgriciDaniel/claude-obsidian (verified all 3 images exist on the public main branch via gh api before swap; sizes 138752, 102970, 175190 bytes). Acceptance: rg -n 'AI-Marketing-Hub/claude-obsidian' across these 8 files returns only matches inside explicit private-mirror disclaimers (the top-of-file callout in install-guide.md, the Pro-member note in CONTRIBUTING.md). Bare canonical references: zero. raw.githubusercontent private-URL refs: zero. Findings closed: F1, F2, F3, F4, F5 from docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md.
…ut (W3) Two parts: 1. Add .obsidian/workspace-visual.json to .gitignore. The file is host-specific UI state (panel positions, last-open file) that creates noisy diffs at clone time and post-edit. workspace-mobile.json was already gitignored on the same rationale; workspace-visual.json had been overlooked. The file was untracked from the index in an earlier auto-commit (bc41e29) before the hook fix landed; this commit ensures it stays untracked. 2. Append the fix-cycle closeout section to docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md. Records: per-finding status table, re-scored dimensions (Security 94/Privacy 94/Data 88/References 96/Files 86; raw avg 91.6; ship verdict GREEN), 4-commit plan, acceptance verification commands, and v1.9.1 + v1.9.2 follow-up rationale for deferred findings (H2, H4, S1-S4, Data M1-M4, References M1-M2, Files M1-M4, W1 wiki/meta relocation). W1 was reassessed and DEFERRED with rationale: 9 of 12 release-session notes have incoming wikilinks; moving them would create 6+ new dead links. The Privacy agent's "intentional case-study material" framing was the right lens. v1.9.2 will handle relocation alongside the wiki dead-link cleanup (References M1+M2) so the wikilink graph is updated in one pass. W2 was reassessed: 3 of 4 flagged files contained only legitimate author self-reference (agricidaniel.com blog, @AgriciDaniel YouTube, github.com/AgriciDaniel profile — canonical public identity, not leaks). The 4th file (wiki/canvases/youtube-explainer.canvas) had 2 AI-Marketing-Hub private-URL refs that were swapped under the canonical-URL cascade (committed via the auto-commit hook as bf4d94b + d2afcbf). Findings closed: W3. Findings deferred-with-rationale: W1 (v1.9.2), W2 reassessed as non-leak.
… (H4+S2) Two safety polish items from v1.9.0 audit closure. H4 — stale-lock reaper between sessions: SessionStart now runs `bash scripts/wiki-lock.sh clear-stale --max-age 3600` on every startup/resume. Locks orphaned by a crashed batch ingest get reaped automatically on the next session, not just on operator demand. The 3600s default is intentionally conservative (matches the admin reaper threshold per wiki-lock.sh header docs) so it cannot accidentally release a lock held by another in-flight session. S2 — opt-out gate for PostToolUse auto-commit: The hook now exits early if `.vault-meta/auto-commit.disabled` exists. Default behavior unchanged for existing users; per-vault opt-out is one `touch` away. Useful for shared repos, CI runs, or any scenario where the operator wants to commit manually. Verification: JSON parses; live test of clear-stale on a missing LOCK_DIR is a no-op via ensure_dirs(); opt-out gate test: `touch .vault-meta/auto-commit.disabled` then trigger a Write — no auto-commit fires. Findings closed: H4, S2 from docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md.
Two lock-primitive hardening items. Data M3 — symlink canonicalization in validate_path(): Previously the function rejected literal `..` segments and absolute paths, but did not canonicalize symlinks. A symlink inside wiki/ resolving outside VAULT_ROOT could escape the lock-namespace check and (combined with downstream filesystem operations) reach outside the vault. Now resolves via python3 os.path.realpath and rejects any path whose canonical form is outside commonpath(VAULT_ROOT, target). Cross-platform: uses python3 not realpath because GNU coreutils and macOS BSD `realpath` have incompatible flag semantics. Only runs the check if python3 is on PATH (graceful no-op fallback otherwise — the literal `..` reject is the floor). Live test: `ln -s /tmp/foo ./escape && bash scripts/wiki-lock.sh acquire escape/x.md` correctly errors with "path resolves outside vault via symlink: escape/x.md". Data M4 — `.vault-meta/locks/.gitkeep`: The directory existed lazily via `ensure_dirs()` on first acquire, which meant fresh clones had no visible locks/ dir until the first wiki-lock invocation. Discoverability gap, not a correctness one. Two changes: 1. `.gitignore` pattern changed from `.vault-meta/locks/` (which excludes the directory itself, preventing negation patterns from re-including children) to `.vault-meta/locks/*` (excludes only contents). This is the documented git workaround for the "cannot re-include a file if its parent is excluded" constraint. 2. `!.vault-meta/locks/.gitkeep` whitelist + the .gitkeep file itself (with a comment explaining the purpose). Net: .lock files still ignored, .gitkeep tracked, dir visible on fresh clone. Findings closed: Data M3, Data M4 from docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md.
…ost assert (Data M1+S4) Data M1 — rerank.py warning routes to hook.log: When the embed-cache lock is unavailable after 3 tries, the WARN line now also appends to .vault-meta/hook.log with a UTC timestamp, in addition to the existing stderr line. Stderr alone is invisible to most callers (subprocess captures, hook contexts); hook.log is the canonical operator-visible log already used by the auto-commit hook. The append-to-hook.log is wrapped in `try: ... except OSError: pass` so a logging failure never blocks the rerank from completing — the temp + os.replace pattern provides write atomicity even without the embed-cache lock or the audit log. S4 — setup-retrieve.sh refuses off-localhost ollama: If $OLLAMA_URL is overridden to point off-localhost, refuse to probe unless --allow-remote-ollama is passed. Mirrors the existing scripts/tiling-check.py:351 gate. Closes a defense-in-depth gap where a malicious env var could redirect probes to an attacker-controlled endpoint. The case statement allows http://127.0.0.1:*, http://localhost:*, and http://[::1]:* without consent; everything else falls through to the consent check. Verification: rerank.py compiles; setup-retrieve.sh bash -n clean; grep confirms both gates in place. Findings closed: Data M1, S4 from docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md.
…ANGELOG (H2+S3) SECURITY.md gains a "Threat model: single-tenant vault" section documenting three intentional design choices and the mitigations for shared-host deployments: 1. wiki-lock.sh release is unconditional (acquire/release typically from separate bash invocations on the same host; PID-binding would fail in normal use). Shared-host mitigation: restrict filesystem permissions on .vault-meta/locks/ to the vault owner. 2. PostToolUse auto-commit hook runs as the user invoking Claude Code and commits wiki/, .raw/, .vault-meta/ to the local repo on every Write/Edit. Per-vault opt-out: touch .vault-meta/auto-commit.disabled. 3. Cross-process resource access (lockfiles, transport snapshots, embed cache) governed by filesystem permissions, not application- layer identity. Standard Linux/macOS permissions are the trust boundary. Closes the documentation gap the v1.9.0 audit flagged: the design choices are correct for single-tenant, but operators in shared-host or shared-CI scenarios deserve explicit notice. Folds together H2 (cross-process lock release documentation) and S3 (multi-tenant caveat) into one coherent section. Version bump 1.9.0 → 1.9.1 across plugin.json + marketplace.json. CHANGELOG entry summarizes the 6 closed HIGH/MEDIUM + 3 closed LOW items, with rescore (91.6 → ~94 raw average) and explicit list of items deferred to v1.9.2 (Data M2 GC, W1 wiki/meta relocation, S1 Excalidraw checksum, GROW note 7th verifier cut). Findings closed: H2, S3 from docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md.
Pre-push verifier on the v1.9.1 audit-closure delta returned FIX-AND-SHIP with 2 MEDIUM findings. Both closed in this commit. V1 — IPv6 loopback bash glob pattern was broken: `http://[::1]:*` in a case statement treats `[::1]` as a POSIX character class (matching a single character that is `:` or `1`), not a literal URL prefix. Legit IPv6 loopback URLs would have fallen through to the "remote, require --allow-remote-ollama" branch — usability bug, not a security regression (safe default still safe). Fix: quote the bracket-bearing pattern as `"http://[::1]:"*`. Verified all three localhost variants (127.0.0.1, localhost, [::1]) now match LOCAL; non-localhost (203.0.113.5) still matches REMOTE. V2 — README static badges still at v1.9.0: 5 occurrences in README.md (release badge, Two-versions callout latest, badge-pinning footnote, architecture-tree skill count, references section). The v1.9.1 version bump in plugin.json + marketplace.json + CHANGELOG.md didn't cascade to the static README badges that don't get their data live from shields.io. Fix: bump all 5 to v1.9.1. This is precisely the class of bug the v1.9.0 audit GROW note proposed adding to agents/verifier.md as a 7th always- check cut (cross-file string consistency). Catching it pre-push instead of post-push confirms the proposed verifier upgrade has real value; that upgrade is still tracked for v1.9.2. Verifier-deferred findings (V3 substring grep tightening, V4 TOCTOU comment) ride into v1.9.2. Verification: bash -n + live case-pattern test for V1; rg "v1\.9\.0" README.md returns empty for V2; make test green.
…tness (v1.9.2) Prompt caching (the only Anthropic API call site): - attach cache_control only above the Haiku 4.5 floor (HAIKU_CACHE_MIN_CHARS=16384, ~4096 tokens); below it the API ignores the marker, so the prior unconditional marker was a no-op that misled the reader. Extracted as pure cache_control_for(). - add per-call cache telemetry (cache_creation/read_input_tokens; integers only, never page content -- preserves the v1.7.1 data-egress posture). - document the sequential-loop invariant that cache reads depend on. Path-handling robustness: - explicit missing page path now exits 3 (was: silent exit 0); out-of-vault path exits 2 with a message (was: a raw ValueError traceback). --all unaffected. - remove dead EXIT_NO_ADDRESS constant + docstring; rename shadowed prefix->progress. Tests/docs: - replace a tautological test with a mocked payload-shape integration test (cache_control by body size + one-line truncation). make test = 9 suites green. - CHANGELOG [1.9.2]; bump plugin.json + marketplace.json to 1.9.2; README badge + early-access version + CONTRIBUTING suite count updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Flip README/docs framing so the public build (AgriciDaniel/claude-obsidian) is the default install everywhere; reposition AI Marketing Hub Pro as early-access to in-development features rather than the default. - marketplace.json -> public form: add $schema, name agricidaniel-claude-obsidian, drop "private mirror / early access" text, remove load-ignored recommendedCompanions field. Passes `claude plugin validate` clean. - Correct the plugin install slug to claude-obsidian@agricidaniel-claude-obsidian everywhere (matches marketplace.json name; verified against `claude plugin list`, which shows slugs derive from the marketplace name field, not owner-repo casing). Pro swap notes invert to claude-obsidian@ai-marketing-hub-claude-obsidian. - README: live Release + CI badges on AgriciDaniel public; two-ways callout, Quick Start, Option 2, FAQ, uninstall all present public as default; Skool Pro link added; test-suite count fixed (8 -> 9); SECURITY.md wording clarified. - install-guide.md, CONTRIBUTING.md, .cursor/.windsurf/.github copilot configs: public canonical as primary; PRs target public. - Add SSS+ files: CITATION.cff, PRIVACY.md, CODEOWNERS, .github/FUNDING.yml. - Neutralize internal tone in .raw ecosystem research. - SEO/GEO pass (from RESEARCH-MASTER, DataForSEO-backed): H1 leads with "Self-Organizing AI Second Brain"; intro captures "AI second brain", "AI notetaker", "AI note-taking", "PKM", "Notion alternative" naturally; 4 verbatim GEO Q&As added to the FAQ (best AI second brain / build a second brain with AI / connect Claude to Obsidian / Notion alternative) for AI Overview + Perplexity citation. No keyword stuffing; brand "claude obsidian" already ranks #1. Gates: secret scan clean (tree + 55-commit history), make test 9/9 suites, plugin validate clean, fresh verifier agent SHIP (0 BLOCKER / 0 HIGH). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Branded pixel-arcade social preview (GitHub recommended 1280x640, 360KB) derived from the existing cover. Upload via repo Settings -> Social preview. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
wiki/index.mdwith all new entrieswiki/log.mdNew pages
Reviewer notes
Personal knowledge base content — no code changes, wiki markdown only.
🤖 Generated with Claude Code