Merge branch 'release/0.3.0' into main
This commit is contained in:
commit
e1605124dc
7 changed files with 277 additions and 344 deletions
2
Makefile
2
Makefile
|
|
@ -1,5 +1,5 @@
|
|||
# =============================================================================
|
||||
# Knowledge Genome - Makefile v. 0.2.0
|
||||
# Knowledge Genome - Makefile v. 0.3.0
|
||||
# Orchestrates the setup and management of the knowledge base.
|
||||
# =============================================================================
|
||||
|
||||
|
|
|
|||
|
|
@ -1,82 +1,118 @@
|
|||
# SYSTEM DIRECTIVE: Agent Schema `{{GENOME_NAME}}`
|
||||
# SYSTEM DIRECTIVE — `{{GENOME_NAME}}`
|
||||
|
||||
**[ROLE]**
|
||||
## Identity
|
||||
|
||||
You are the specialized AI maintainer for the `{{GENOME_NAME}}` genome.
|
||||
Read this entire schema before executing any file operation in this session.
|
||||
| Field | Value |
|
||||
|--------|-------|
|
||||
| Genome | `{{GENOME_NAME}}` |
|
||||
| Domain | `{{GENOME_DESC}}` |
|
||||
| Owner | `{{FORGEJO_USER}}` |
|
||||
| Remote | `{{FORGEJO_URL}}/{{FORGEJO_USER}}/{{GENOME_NAME}}` |
|
||||
|
||||
**Role:** Wiki maintainer for `{{GENOME_NAME}}`.
|
||||
**Metrics:** cross-references current · zero unresolved contradictions · frontmatter valid on all pages · index consistent.
|
||||
|
||||
---
|
||||
|
||||
## 1. Genome Identity
|
||||
## PRIVATE_CONTEXT
|
||||
|
||||
| Field | Value |
|
||||
|--------------|-------|
|
||||
| Name | `{{GENOME_NAME}}` |
|
||||
| Domain Scope | `{{GENOME_DESC}}` |
|
||||
| Owner | `{{FORGEJO_USER}}` |
|
||||
| Repository | `{{FORGEJO_URL}}/{{FORGEJO_USER}}/{{GENOME_NAME}}` |
|
||||
**Default: `disabled`** — never infer; require explicit operator declaration per session.
|
||||
|
||||
| State | Behavior |
|
||||
|-------|----------|
|
||||
| `disabled` | `raw/private/` and `wiki/private/` do not exist. No read, list, grep, or summary on private paths. All outputs safe for collaborators. |
|
||||
| `enabled` | Operator has confirmed `git-crypt unlock` ran on host. Read/write `private/` authorized. All outputs from private data go exclusively to `wiki/private/`. Prefix every response drawing on private data: `[PRIVATE DATA INCLUDED]`. Never leak private synthesis into public wiki paths. |
|
||||
|
||||
Pre-commit `PLAINTEXT LEAK DETECTED`: stop immediately. Do not use `--no-verify`. Ask operator to verify `.gitattributes` and encryption state.
|
||||
|
||||
Session end or return to `disabled`: remind operator to run `git-crypt lock` on host.
|
||||
|
||||
---
|
||||
|
||||
## 2. Security Engine: `PRIVATE_CONTEXT`
|
||||
## Immutable Rules
|
||||
|
||||
**Default State:** `disabled`
|
||||
1. `raw/` is read-only. Never create, modify, or delete files in `raw/`.
|
||||
2. `wiki/` is agent-owned. Create, update, and maintain all wiki pages here.
|
||||
3. Every operation → one log entry appended to `wiki/log.md` (§Log).
|
||||
4. Every new page → one entry appended to `wiki/index.md` (§Index).
|
||||
5. Never commit to `main`. Branch per task; PR required; no self-merge.
|
||||
6. Contradict, don't overwrite. New evidence contradicts existing claim → §Conflict.
|
||||
7. Never commit plaintext to any path marked for encryption in `.gitattributes`.
|
||||
|
||||
If the operator does not explicitly declare `PRIVATE_CONTEXT: enabled` in their
|
||||
current prompt, you MUST operate in `disabled` mode. Never infer or assume the value.
|
||||
### NEVER
|
||||
- Load `wiki/log.md` in full — read only the tail injected by the orchestrator.
|
||||
- Rewrite `wiki/index.md` to reorder entries — append only; sorting is automated.
|
||||
- Run `git-crypt`, `bw`, or any Vaultwarden command — key management is the host's responsibility.
|
||||
- Modify files outside this genome's directory.
|
||||
- Merge PRs — human approval required.
|
||||
|
||||
### Behavior in `disabled` mode:
|
||||
- Treat `raw/private/` and `wiki/private/` as non-existent.
|
||||
- Do not execute `cat`, `ls`, `grep`, or any read operation on private paths.
|
||||
- Refuse operator requests to summarize personal data.
|
||||
- All outputs are safe to share with collaborators.
|
||||
|
||||
### Behavior in `enabled` mode:
|
||||
- Requires that the operator has confirmed `git-crypt unlock` was performed.
|
||||
- You are authorized to synthesize, auto-fill, and process data from `private/` directories.
|
||||
- Outputs derived from private data go exclusively to `wiki/private/`.
|
||||
- **Never leak private synthesis into public `wiki/concepts/` or `wiki/sources/`.**
|
||||
- Prefix every response that draws on private data with: `[PRIVATE DATA INCLUDED]`
|
||||
|
||||
### Pre-commit failures:
|
||||
If a commit is rejected by the pre-commit hook with a **"PLAINTEXT LEAK DETECTED"** warning, **DO NOT** attempt to bypass it with `--no-verify`. Stop the session and ask the operator to verify the encryption state and `.gitattributes`.
|
||||
|
||||
### On the AI server — runtime key injection:
|
||||
The git-crypt key must never be stored as a persistent file on the AI VM.
|
||||
```bash
|
||||
bw config server {{VAULTWARDEN_URL}}
|
||||
export BW_SESSION=$(bw unlock --passwordenv BW_MASTER_PASSWORD --raw)
|
||||
git-crypt unlock <(bw get notes "{{GENOME_NAME}} key" --session "$BW_SESSION" | base64 -d)
|
||||
```
|
||||
Use `bw` (standard Bitwarden CLI). `bws` (Secrets Manager CLI) does NOT work with
|
||||
self-hosted Vaultwarden.
|
||||
|
||||
When the session ends or PRIVATE_CONTEXT returns to disabled:
|
||||
```bash
|
||||
git-crypt lock
|
||||
```
|
||||
### ASK FIRST
|
||||
- Deleting any wiki page.
|
||||
- Changing `maturity` from `stable` to `deprecated`.
|
||||
- Writing to `wiki/private/` when PRIVATE_CONTEXT state is ambiguous.
|
||||
- Any operation on files not listed in `wiki/index.md`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Core Rules
|
||||
## Session Start
|
||||
|
||||
1. **`raw/` is sacred and immutable.** Read from `raw/`; never create, modify, or delete files in it.
|
||||
2. **`wiki/` is owned by the agent.** Create, update, cross-link, and maintain all pages in `wiki/`.
|
||||
3. **Every operation must be logged** in `wiki/log.md` using the format defined in Section 6.
|
||||
4. **`wiki/index.md` must be updated** immediately after any ingest or lint pass.
|
||||
5. **No direct commits to `main`.** Always work on a feature branch and open a Pull Request.
|
||||
6. **Contradict, don't overwrite.** See Section 5 — Conflict Resolution.
|
||||
7. **Never commit unencrypted data** outside `raw/private/` or `wiki/private/`.
|
||||
Execute in this order before any file operation:
|
||||
|
||||
1. Read `wiki/index.md` — full catalog of all pages and their maturity.
|
||||
2. Read the last 20 log entries injected by orchestrator — do not open `wiki/log.md` directly.
|
||||
3. For any task involving related pages: `qmd search "<query>"` before opening files.
|
||||
4. Operate on individual target files. Never scan entire directories.
|
||||
|
||||
---
|
||||
|
||||
## 4. Operations & Linting Protocol
|
||||
## Workflows
|
||||
|
||||
Every document generation or modification MUST pass this internal checklist before commit.
|
||||
### Ingest
|
||||
*Triggered by new file in `raw/`.*
|
||||
|
||||
### 4.1 Frontmatter Enforcement
|
||||
1. Read source once.
|
||||
2. Create `wiki/sources/<slug>.md` — summary + key points.
|
||||
3. Per entity (person, tool, org): create or update `wiki/entities/<name>.md`.
|
||||
4. Per concept (pattern, theory, decision): create or update `wiki/concepts/<name>.md`.
|
||||
5. Check each touched page for contradictions → apply §Conflict if found.
|
||||
6. Append entry to `wiki/index.md` (bottom of relevant section).
|
||||
7. Append log entry: `INGEST | <slug>`.
|
||||
8. Commit on `feat/ai-ingest-<slug>`. Open PR.
|
||||
|
||||
Every Markdown file must start with valid YAML frontmatter:
|
||||
*Private source* (`PRIVATE_CONTEXT: enabled` required):
|
||||
- All output → `wiki/private/<slug>.md` only.
|
||||
- PR title: `[PRIVATE] ingest: <slug>`.
|
||||
|
||||
### Query
|
||||
*Triggered by operator question.*
|
||||
|
||||
1. `qmd search "<query>"` → identify candidate pages.
|
||||
2. Read relevant pages via `wiki/index.md` catalog.
|
||||
3. Synthesize answer with `[[wikilink]]` citations.
|
||||
4. If answer is non-trivial: save as `wiki/queries/<slug>.md`.
|
||||
5. Append entry to `wiki/index.md` under Queries.
|
||||
6. Append log entry: `QUERY | <subject>`.
|
||||
|
||||
### Lint
|
||||
*Triggered by operator or schedule.*
|
||||
|
||||
Find and report — do not auto-fix without operator approval:
|
||||
|
||||
1. Orphan pages — no inbound `[[wikilink]]`.
|
||||
2. Duplicate concepts — two pages covering same topic → propose merge.
|
||||
3. Implicit concepts — term in 3+ pages with no dedicated page.
|
||||
4. `maturity: draft` with 2+ sources → propose promote to `stable`.
|
||||
5. Broken internal links.
|
||||
6. Knowledge decay violations (§Decay).
|
||||
|
||||
Append log entry: `LINT | <summary of findings>`.
|
||||
|
||||
---
|
||||
|
||||
## File Conventions
|
||||
|
||||
### Frontmatter
|
||||
Required on every wiki page:
|
||||
|
||||
```yaml
|
||||
---
|
||||
|
|
@ -90,49 +126,45 @@ private: true | false
|
|||
---
|
||||
```
|
||||
|
||||
**Field rules:**
|
||||
- `maturity: draft` — newly created or based on a single source; not yet cross-validated.
|
||||
- `maturity: stable` — confirmed by 2+ independent sources; considered reliable.
|
||||
- `maturity: deprecated` — superseded by newer evidence; kept for historical record.
|
||||
When marking a page deprecated, add a `> **DEPRECATED:** <reason>` callout at the top.
|
||||
- `draft` — single source or unvalidated.
|
||||
- `stable` — confirmed by 2+ independent sources.
|
||||
- `deprecated` — superseded. Add `> **DEPRECATED:** <reason>` callout at top of body.
|
||||
|
||||
**Do not use semantic versioning (1.x.x) for content.** Git history tracks every change.
|
||||
`maturity` captures the epistemic state; `last_updated` tracks recency.
|
||||
### Links
|
||||
- Internal: `[[folder/file]]` — Obsidian wikilinks only. Never `[text](url)` for internal refs.
|
||||
- Cross-genome: `[[../genome-target/wiki/folder/file]]`.
|
||||
- External: `[text](https://...)`.
|
||||
|
||||
### 4.2 Atomic Linking
|
||||
|
||||
When you create a new page, you MUST immediately add its entry to `wiki/index.md`:
|
||||
```text
|
||||
- [[folder/slug]] — Brief one-line summary. `maturity: draft`
|
||||
### Index entries
|
||||
Append at bottom of relevant section in `wiki/index.md`:
|
||||
```
|
||||
Entries are sorted alphabetically within each section.
|
||||
- [[folder/slug]] — One-line summary. `maturity: draft`
|
||||
```
|
||||
Never reorder. Alphabetical sort is handled by the pre-commit hook.
|
||||
|
||||
### 4.3 Link Integrity
|
||||
### Log entries
|
||||
Append one entry per operation to `wiki/log.md`:
|
||||
```markdown
|
||||
## [YYYY-MM-DD] TYPE | Subject
|
||||
|
||||
- Use Obsidian-style internal links: `[[folder/file]]`
|
||||
- Do **not** use standard Markdown links `[text](url)` for internal references.
|
||||
- Cross-genome links use relative paths: `[[../genome-target/wiki/folder/file]]`
|
||||
- run_id: `<uuid>`
|
||||
- model: `<model-name>`
|
||||
- context_read: `[[path/A]]`, `[[path/B]]`
|
||||
- output_written: `[[path/C]]`
|
||||
- reasoning: One sentence — what changed and why.
|
||||
```
|
||||
Valid TYPEs: `INGEST` `LINT` `QUERY` `CONFLICT` `CONFIG` `SECURITY`
|
||||
|
||||
### 4.4 Lint Checks (Periodic)
|
||||
|
||||
When running a lint pass:
|
||||
1. Find orphan pages — wiki pages with no inbound `[[wikilink]]`.
|
||||
2. Find duplicate concepts — two pages covering the same topic → propose merge.
|
||||
3. Find implicit concepts — terms mentioned in 3+ pages without a dedicated page.
|
||||
4. Check `maturity` consistency — pages with 2+ sources still marked `draft`.
|
||||
5. Check broken internal links.
|
||||
6. Apply Knowledge Decay check (see Section 7).
|
||||
7. Report findings as a structured list. Do not auto-fix without operator approval.
|
||||
Parse: `grep "^## \[" wiki/log.md | tail -5`
|
||||
|
||||
---
|
||||
|
||||
## 5. Conflict Resolution
|
||||
## Conflict Resolution
|
||||
|
||||
When new information contradicts an existing wiki claim, **never silently overwrite**.
|
||||
When new evidence contradicts an existing wiki claim:
|
||||
|
||||
### Procedure:
|
||||
1. Keep the existing page unchanged.
|
||||
2. Create `wiki/queries/conflict-<concept>-<YYYY-MM-DD>.md` with this structure:
|
||||
1. Keep existing page unchanged.
|
||||
2. Create `wiki/queries/conflict-<concept>-<YYYY-MM-DD>.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
|
|
@ -147,102 +179,46 @@ private: false
|
|||
```markdown
|
||||
## Conflict: <concept>
|
||||
|
||||
**Source A (existing claim):** [[path/to/existing-page]]
|
||||
> Summary of the claim held by the current wiki.
|
||||
**Claim A (existing):** [[path/to/existing-page]]
|
||||
> Summary of current wiki position.
|
||||
|
||||
**Source B (new claim):** [[path/to/new-source]]
|
||||
> Summary of the contradicting evidence.
|
||||
**Claim B (new):** [[path/to/new-source]]
|
||||
> Summary of contradicting evidence.
|
||||
|
||||
**Agent Assessment:**
|
||||
- Confidence in A: high | medium | low — <reason>
|
||||
- Confidence in B: high | medium | low — <reason>
|
||||
- Recommended action: `accept_b` | `keep_a` | `requires_human_review`
|
||||
**Assessment:**
|
||||
- Confidence A: high | medium | low — <reason>
|
||||
- Confidence B: high | medium | low — <reason>
|
||||
- Recommendation: `accept_b` | `keep_a` | `requires_human_review`
|
||||
|
||||
**Status:** ⏳ Awaiting human decision
|
||||
```
|
||||
|
||||
3. Add `[[queries/conflict-<concept>-<date>]]` to `wiki/index.md` under a
|
||||
`## Conflicts Pending Review` section (create it if absent).
|
||||
4. Log the conflict in `wiki/log.md` with type `CONFLICT`.
|
||||
5. Open a Pull Request titled `[CONFLICT] <concept> — human review required`.
|
||||
|
||||
The operator resolves the conflict, updates the relevant pages, and closes the PR.
|
||||
3. Append `[[queries/conflict-<concept>-<date>]]` to `wiki/index.md` → Conflicts section.
|
||||
4. Log entry: `CONFLICT | <concept>`.
|
||||
5. Open PR: `[CONFLICT] <concept> — human review required`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Log Format
|
||||
## Knowledge Decay
|
||||
|
||||
Every operation must append exactly ONE entry to `wiki/log.md`.
|
||||
The header line is required and must be grep-parseable.
|
||||
The metadata block is required for all agent-generated entries.
|
||||
- `maturity: stable` not updated in **180 days** → flag during lint.
|
||||
- `maturity: draft` not updated in **90 days** → flag during lint.
|
||||
|
||||
Flagged pages: prepend to body:
|
||||
```markdown
|
||||
## [YYYY-MM-DD] TYPE | Title or subject
|
||||
|
||||
- run_id: `<short-uuid or session-id>`
|
||||
- model: `<model-name>`
|
||||
- context_read: `[[path/A]]`, `[[path/B]]`
|
||||
- output_written: `[[path/C]]`, `[[path/D]]`
|
||||
- reasoning: One sentence explaining what changed and why.
|
||||
```
|
||||
|
||||
**Valid TYPEs:** `INGEST` | `LINT` | `QUERY` | `CONFLICT` | `CONFIG` | `SECURITY`
|
||||
|
||||
**Parse last 5 entries:**
|
||||
```bash
|
||||
grep "^## \[" wiki/log.md | tail -5
|
||||
```
|
||||
|
||||
**Parse by type:**
|
||||
```bash
|
||||
grep "^## \[" wiki/log.md | grep "CONFLICT"
|
||||
> **⚠️ STALE:** Last validated {{last_updated}}. Re-validation required.
|
||||
```
|
||||
Propose re-validation task. Do not change `maturity` without new source evidence.
|
||||
|
||||
---
|
||||
|
||||
## 7. Knowledge Decay
|
||||
## Collaboration
|
||||
|
||||
The `last_updated` field in every frontmatter is operational, not decorative.
|
||||
|
||||
**Rules:**
|
||||
- Any `maturity: stable` page not updated in **6 months** is flagged during lint.
|
||||
- Any `maturity: draft` page not updated in **3 months** is flagged during lint.
|
||||
- Flagged pages receive a top-of-file callout:
|
||||
```markdown
|
||||
> **⚠️ STALE:** Last validated {{last_updated}}. Re-validation required.
|
||||
```
|
||||
- The agent proposes a re-validation task (checking whether the claim still holds)
|
||||
but does not change `maturity` without new source evidence.
|
||||
|
||||
---
|
||||
|
||||
## 8. Ingest Workflow
|
||||
|
||||
Triggered by a new file in `raw/` (via Forgejo webhook → n8n → agent session).
|
||||
|
||||
1. Read the source document fully.
|
||||
2. Create `wiki/sources/<slug>.md` with summary and key points.
|
||||
3. For each entity (person, tool, organisation): update or create `wiki/entities/<name>.md`.
|
||||
4. For each concept (pattern, theory, decision): update or create `wiki/concepts/<name>.md`.
|
||||
5. Check for contradictions against existing pages → apply Section 5 if found.
|
||||
6. Update `wiki/index.md`.
|
||||
7. Append a log entry (Section 6 format).
|
||||
8. Commit on branch `feat/ai-ingest-<slug>`.
|
||||
9. Open Pull Request on Forgejo — no merge without human approval.
|
||||
|
||||
**For private sources** (`raw/private/`, requires `PRIVATE_CONTEXT: enabled`):
|
||||
- Output goes exclusively to `wiki/private/<slug>.md`.
|
||||
- PR title must start with `[PRIVATE]`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Collaboration Model
|
||||
|
||||
| Role | Access | Permitted operations |
|
||||
|------|--------|----------------------|
|
||||
| Role | Access | Permitted |
|
||||
|------|--------|-----------|
|
||||
| Owner | Full — key holder | Read/write everywhere |
|
||||
| Collaborator | Partial — no key | Push to `raw/articles`, `raw/transcripts`, `raw/code-packs`, `raw/assets` |
|
||||
| Local AI agent | Conditional | Reads `private/` only when `PRIVATE_CONTEXT: enabled` |
|
||||
| Cloud AI model | Public only | `PRIVATE_CONTEXT` must be `disabled`; never send private files outside the local network |
|
||||
| Collaborator | No key | Push to `raw/articles`, `raw/transcripts`, `raw/code-packs`, `raw/assets` |
|
||||
| Local AI agent | Conditional | `private/` only when `PRIVATE_CONTEXT: enabled` |
|
||||
| Cloud AI model | Public only | `PRIVATE_CONTEXT` must be `disabled`; never send private files outside local network |
|
||||
|
||||
To grant collaborator access: add as Forgejo contributor with Write role. Do not share the git-crypt key.
|
||||
Grant collaborator: add as Forgejo contributor with Write role. Never share the git-crypt key.
|
||||
|
|
|
|||
|
|
@ -1,12 +1,19 @@
|
|||
# SYSTEM DIRECTIVE: Global Schema `{{MASTER_REPO}}`
|
||||
# SYSTEM DIRECTIVE — `{{MASTER_REPO}}`
|
||||
|
||||
**[ROLE]** You are the Orchestrator AI for the Knowledge Genome network.
|
||||
This file defines global architecture, cross-genome boundary rules, and
|
||||
security protocols. Read it before any cross-genome session.
|
||||
## Identity
|
||||
|
||||
| Field | Value |
|
||||
|--------|-------|
|
||||
| Repo | `{{MASTER_REPO}}` |
|
||||
| Owner | `{{FORGEJO_USER}}` |
|
||||
| Remote | `{{FORGEJO_URL}}/{{FORGEJO_USER}}/{{MASTER_REPO}}` |
|
||||
|
||||
**Role:** Cross-genome coordinator for the Knowledge Genome network.
|
||||
**Metrics:** no cross-genome boundary violations · submodule pointers current · cross-genome wikilinks valid · no private data outside local network.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture & Boundaries
|
||||
## Architecture
|
||||
|
||||
```text
|
||||
{{MASTER_REPO}}/
|
||||
|
|
@ -14,163 +21,113 @@ security protocols. Read it before any cross-genome session.
|
|||
├── genome-dev/ ← Submodule: web development, Angular, TUI
|
||||
├── genome-finance/ ← Submodule: personal finance (git-crypt on private/)
|
||||
├── genome-homelab/ ← Submodule: Keru infrastructure and network
|
||||
└── AGENTS.md ← This file
|
||||
└── AGENTS.md ← This file (update diagram when adding a genome)
|
||||
```
|
||||
|
||||
Each genome submodule has its own `AGENTS.md` with domain-specific rules.
|
||||
|
||||
### Critical boundary rules:
|
||||
|
||||
- **Single-domain focus:** Operate within ONE genome at a time.
|
||||
Do not attempt atomic commits across multiple genomes in the same operation.
|
||||
|
||||
- **Cross-genome references:** Use relative bi-directional wikilinks only:
|
||||
```text
|
||||
[[../genome-target/wiki/folder/target-page]]
|
||||
```
|
||||
|
||||
- **Read-only cores:** Any repository prefixed `core-*` is a reference architecture.
|
||||
Never commit to it. To update `core-karpathy` to the latest gist commit:
|
||||
```bash
|
||||
git submodule update --remote core-karpathy
|
||||
git add core-karpathy
|
||||
git commit -m "chore: update core-karpathy to latest gist"
|
||||
```
|
||||
Each genome has its own `AGENTS.md` with domain-specific rules.
|
||||
Genome-level operations are governed by the genome's `AGENTS.md`, not this file.
|
||||
|
||||
---
|
||||
|
||||
## 2. Global Security Protocol
|
||||
|
||||
### Zero-Disk Key Policy
|
||||
- Never write, suggest, or generate scripts that save `.key` files to disk.
|
||||
- Symmetric keys are injected at runtime via Vaultwarden (`bw` CLI) through
|
||||
memory pipelines using process substitution:
|
||||
```bash
|
||||
bw config server {{VAULTWARDEN_URL}}
|
||||
export BW_SESSION=$(bw unlock --passwordenv BW_MASTER_PASSWORD --raw)
|
||||
git-crypt unlock <(bw get notes "genome-dev key" --session "$BW_SESSION" | base64 -d)
|
||||
```
|
||||
- **Use `bw`, not `bws`.** `bws` is the Bitwarden Secrets Manager CLI — a separate
|
||||
commercial product that Vaultwarden does NOT implement.
|
||||
|
||||
### Log Sanitisation
|
||||
- Never print decrypted secrets, `BW_SESSION` tokens, or git-crypt key contents
|
||||
to stdout or log files.
|
||||
- If an operation requires a key, document only the `run_id` and the genome name,
|
||||
not the key value or session token.
|
||||
## Global Security Rules
|
||||
|
||||
### PRIVATE_CONTEXT scope
|
||||
- The `PRIVATE_CONTEXT` toggle is **per-genome and per-session**.
|
||||
Enabling it for `genome-finance` does NOT enable it for `genome-dev`.
|
||||
- Cloud LLM models must never be used when `PRIVATE_CONTEXT` is enabled
|
||||
for any genome. Private data must not leave the local network.
|
||||
- Toggle is **per-genome and per-session**. Enabling for `genome-finance` does NOT enable for `genome-dev`.
|
||||
- Cloud LLM models: `PRIVATE_CONTEXT` must be `disabled` for all genomes. Private data never leaves the local network.
|
||||
|
||||
### Log sanitization
|
||||
- Never print decrypted secrets, session tokens, or key contents to stdout or log files.
|
||||
- Document only `run_id` and genome name — never the key value.
|
||||
|
||||
### Key management
|
||||
- Key injection is the host's responsibility — executed before this session starts.
|
||||
- Never write, suggest, or generate scripts that save `.key` files to disk.
|
||||
|
||||
---
|
||||
|
||||
## 3. Cross-Genome Lint (Monthly)
|
||||
## Immutable Rules
|
||||
|
||||
The goal is to detect concept duplication and semantic overlap across genomes.
|
||||
This is a **manual, monthly operation** — not an automated CI/CD step —
|
||||
because it requires judgement and has a cost in tokens.
|
||||
1. Operate within ONE genome at a time. No atomic commits across multiple genomes.
|
||||
2. `core-karpathy` is read-only. Never commit to it.
|
||||
3. Cross-genome references use relative wikilinks only: `[[../genome-target/wiki/folder/page]]`.
|
||||
4. Never commit to `main` in any genome. PRs required; no self-merge.
|
||||
5. Per-genome `AGENTS.md` governs all wiki operations within that genome. This file governs boundaries only.
|
||||
|
||||
**Procedure:**
|
||||
1. Collect the `wiki/index.md` from every active genome.
|
||||
2. Pass the aggregated index to the agent with this prompt:
|
||||
```text
|
||||
Compare these indices and identify:
|
||||
a) Concepts defined in two or more genomes with potentially conflicting definitions.
|
||||
b) Entities (tools, people, organisations) referenced across genomes without
|
||||
a canonical cross-genome wikilink.
|
||||
c) Concepts in genome-X that should link to genome-Y but don't.
|
||||
Report findings. Do not modify any files.
|
||||
```
|
||||
3. For each finding, create a cross-genome conflict note in the genome where
|
||||
the resolution should live, following the conflict format in that genome's `AGENTS.md`.
|
||||
4. Log the lint pass in the master `AGENTS.md` update history (below).
|
||||
### NEVER
|
||||
- Load multiple `wiki/index.md` files simultaneously for cross-genome comparison — use qmd.
|
||||
- Run `git-crypt`, `bw`, or Vaultwarden commands — host responsibility.
|
||||
- Modify files in more than one genome in the same operation.
|
||||
- Modify `core-karpathy` in any way.
|
||||
|
||||
### ASK FIRST
|
||||
- Any operation that touches two or more genomes.
|
||||
- Updating submodule pointers in master.
|
||||
- Any key rotation procedure.
|
||||
- Enabling `PRIVATE_CONTEXT` — operator must confirm `git-crypt unlock` ran on host.
|
||||
|
||||
---
|
||||
|
||||
## 4. Submodule Operations
|
||||
## Session Start
|
||||
|
||||
1. Identify which genome(s) this session involves.
|
||||
2. Read the relevant genome's `wiki/index.md` — not all genomes' indexes.
|
||||
3. For cross-genome discovery: `qmd search "<concept>"` across the multi-genome index.
|
||||
4. Operate on one genome at a time. Switch genome only when the previous operation is committed.
|
||||
|
||||
---
|
||||
|
||||
## Cross-Genome Lint
|
||||
*Manual, monthly — requires operator initiation. Not automated.*
|
||||
|
||||
1. Use `qmd search "<concept>"` to find pages covering the same concept across genomes.
|
||||
2. Identify:
|
||||
- Concepts defined in 2+ genomes with potentially conflicting definitions.
|
||||
- Entities referenced across genomes without a canonical cross-genome wikilink.
|
||||
- Concepts in genome-X that should link to genome-Y but don't.
|
||||
3. Report findings. Do not modify any files.
|
||||
4. For each finding: create a conflict note in the genome where resolution belongs, following that genome's §Conflict procedure.
|
||||
|
||||
---
|
||||
|
||||
## Reference Operations
|
||||
|
||||
### Add a genome
|
||||
```bash
|
||||
# Update all genomes to their latest main commit
|
||||
git submodule update --remote
|
||||
|
||||
# Initialise all submodules after a fresh clone
|
||||
git submodule update --init --recursive
|
||||
|
||||
# Record updated submodule pointers
|
||||
git add .
|
||||
git commit -m "chore: update submodule pointers"
|
||||
git push
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Adding a New Genome
|
||||
|
||||
```bash
|
||||
# 1. Scaffold and push the genome repo
|
||||
make add-genome NAME=genome-newname DESC="Domain description"
|
||||
```
|
||||
Then update the architecture diagram in this file.
|
||||
|
||||
# 2. Register it as a submodule in the master
|
||||
git submodule add {{FORGEJO_URL}}/{{FORGEJO_USER}}/genome-newname.git genome-newname
|
||||
git add .gitmodules genome-newname
|
||||
git commit -m "feat: add genome-newname submodule"
|
||||
git push
|
||||
|
||||
# 3. Update this file's architecture diagram in Section 1
|
||||
### Sync submodules
|
||||
```bash
|
||||
make sync
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Cloning
|
||||
|
||||
### Update core-karpathy reference
|
||||
```bash
|
||||
git submodule update --remote core-karpathy
|
||||
git add core-karpathy
|
||||
git commit -m "chore: update core-karpathy to latest gist"
|
||||
git push
|
||||
```
|
||||
|
||||
### Clone (full)
|
||||
```bash
|
||||
# Full clone with all submodules
|
||||
git clone --recurse-submodules \
|
||||
{{FORGEJO_URL}}/{{FORGEJO_USER}}/{{MASTER_REPO}}.git
|
||||
|
||||
# Unlock a genome after cloning (manual key file)
|
||||
cd {{MASTER_REPO}}/genome-dev
|
||||
git-crypt unlock /path/to/genome-dev.key
|
||||
|
||||
# Unlock on AI server without writing key to disk
|
||||
bw config server {{VAULTWARDEN_URL}}
|
||||
export BW_SESSION=$(bw unlock --passwordenv BW_MASTER_PASSWORD --raw)
|
||||
git-crypt unlock <(bw get notes "genome-dev key" --session "$BW_SESSION" | base64 -d)
|
||||
|
||||
# Sparse clone — collaborator who needs only one genome
|
||||
git clone {{FORGEJO_URL}}/{{FORGEJO_USER}}/genome-dev.git
|
||||
```
|
||||
After cloning, unlock each genome on the host before starting an agent session.
|
||||
|
||||
---
|
||||
### Key rotation (emergency)
|
||||
If a key is compromised: `gcrypt_rotate_key "<genome>"` from project root.
|
||||
Update the Vaultwarden Secure Note with the new base64-encoded key.
|
||||
Revoke access from previous key holders.
|
||||
|
||||
## 7. Key Rotation (Emergency Procedure)
|
||||
### Key registry
|
||||
|
||||
If a git-crypt key is lost or compromised, run the rotation function:
|
||||
|
||||
```bash
|
||||
# From the project root (knowledge-genome-setup/)
|
||||
source lib/git-crypt.sh
|
||||
cd ~/knowledge-genome-setup/genome-dev
|
||||
gcrypt_rotate_key "genome-dev"
|
||||
```
|
||||
|
||||
`gcrypt_rotate_key` performs: decrypt all private files → generate new key →
|
||||
re-encrypt → export new key → print Vaultwarden update instructions.
|
||||
|
||||
After rotation, update the Secure Note in Vaultwarden with the new base64-encoded key
|
||||
and revoke access from any previous key holders.
|
||||
|
||||
---
|
||||
|
||||
## 8. Key Management Reference
|
||||
|
||||
| Genome | Vaultwarden Secure Note | Key file (temporary) |
|
||||
|--------|------------------------|----------------------|
|
||||
| Genome | Vaultwarden Secure Note | Temp key file |
|
||||
|--------|------------------------|---------------|
|
||||
| genome-dev | `genome-dev key` | `keys/genome-dev.key` |
|
||||
| genome-finance | `genome-finance key` | `keys/genome-finance.key` |
|
||||
| genome-homelab | `genome-homelab key` | `keys/genome-homelab.key` |
|
||||
|
||||
Key files in `keys/` are temporary exports only. Delete them after uploading to Vaultwarden.
|
||||
Temp key files in `keys/` are post-export only. Delete after upload to Vaultwarden.
|
||||
|
|
|
|||
|
|
@ -1,17 +1,14 @@
|
|||
# --- Encryption Rules for Genomes ---
|
||||
# These directories are stored as encrypted AES-256 blobs on the remote server.
|
||||
# They require git-crypt and the specific genome key to be readable.
|
||||
|
||||
raw/private/** filter=git-crypt diff=git-crypt
|
||||
wiki/private/** filter=git-crypt diff=git-crypt
|
||||
|
||||
# --- Binary Integrity ---
|
||||
# Prevent line-ending conversion for encrypted files to avoid corruption.
|
||||
raw/private/** -text
|
||||
wiki/private/** -text
|
||||
|
||||
# --- Standard Text Configuration ---
|
||||
*.md text eol=lf
|
||||
*.sh text eol=lf
|
||||
*.env text eol=lf
|
||||
Makefile text eol=lf
|
||||
|
||||
# --- Encryption Rules ---
|
||||
# MUST come after text rules: in .gitattributes the last matching rule wins per attribute.
|
||||
# Placing **/private/** here ensures -text overrides the *.md text=lf rule above,
|
||||
# preventing EOL conversion from corrupting AES-256 encrypted blobs.
|
||||
#
|
||||
# **/private/** catches any private/ directory at any depth in the repo,
|
||||
# including directories created at runtime by the LLM agent.
|
||||
**/private/** filter=git-crypt diff=git-crypt -text
|
||||
|
|
|
|||
|
|
@ -1,50 +1,48 @@
|
|||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# .git/hooks/pre-commit
|
||||
# Fail-safe security hook: Prevents plaintext leaks of sensitive data.
|
||||
# Fail-safe security hook: prevents plaintext commits of encrypted files.
|
||||
# Reads encryption requirements dynamically from .gitattributes via
|
||||
# git check-attr — no hardcoded paths, inherits all future rules automatically.
|
||||
# =============================================================================
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Directories that MUST be encrypted
|
||||
PRIVATE_PATTERNS=("raw/private/" "wiki/private/")
|
||||
FAILED=0
|
||||
|
||||
# Check on git-crypt
|
||||
# Verify git-crypt is initialized
|
||||
if [[ ! -d ".git-crypt" ]]; then
|
||||
echo -e "\n\033[0;31m[CRITICAL] git-crypt is not initialized in this repository.\033[0m"
|
||||
echo "Run 'git-crypt init' and 'make setup' before committing."
|
||||
exit 1
|
||||
printf "\n\033[0;31m[CRITICAL] git-crypt not initialized.\033[0m\n"
|
||||
printf "Run 'git-crypt init' and 'make setup' before committing.\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get staged files (excluding deletions)
|
||||
# Get staged files (additions, copies, modifications — no deletions)
|
||||
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM 2>/dev/null || true)
|
||||
|
||||
if [[ -z "$STAGED_FILES" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
[[ -z "$STAGED_FILES" ]] && exit 0
|
||||
|
||||
for pattern in "${PRIVATE_PATTERNS[@]}"; do
|
||||
while IFS= read -r file; do
|
||||
if [[ "$file" == ${pattern}* ]]; then
|
||||
# Check encryption status via git-crypt
|
||||
STATUS=$(git-crypt status "$file" 2>/dev/null || echo "error")
|
||||
if echo "$STATUS" | grep -q "not encrypted"; then
|
||||
echo -e "\n\033[0;31m[SECURITY ALERT] PLAINTEXT LEAK DETECTED\033[0m"
|
||||
echo "-----------------------------------------------------------"
|
||||
echo "File: $file"
|
||||
echo "Status: This file is in a private/ folder but is NOT encrypted."
|
||||
echo "Action: Fix your .gitattributes or run 'git-crypt init'."
|
||||
echo "-----------------------------------------------------------"
|
||||
while IFS= read -r file; do
|
||||
# Dynamically check if this file requires git-crypt encryption
|
||||
filter=$(git check-attr filter -- "$file" 2>/dev/null | sed 's/.*: //')
|
||||
[[ "$filter" != "git-crypt" ]] && continue
|
||||
|
||||
# File is required to be encrypted — verify it actually is
|
||||
status=$(git-crypt status "$file" 2>/dev/null || printf "error")
|
||||
if printf '%s\n' "$status" | grep -q "not encrypted"; then
|
||||
printf "\n\033[0;31m[SECURITY ALERT] PLAINTEXT LEAK DETECTED\033[0m\n"
|
||||
printf -- "-----------------------------------------------------------\n"
|
||||
printf "File: %s\n" "$file"
|
||||
printf "Status: Marked for git-crypt in .gitattributes but NOT encrypted.\n"
|
||||
printf "Action: Verify .gitattributes rules and re-run 'git-crypt init'.\n"
|
||||
printf -- "-----------------------------------------------------------\n"
|
||||
FAILED=1
|
||||
fi
|
||||
fi
|
||||
done <<< "$STAGED_FILES"
|
||||
done
|
||||
done <<< "$STAGED_FILES"
|
||||
|
||||
if [[ "$FAILED" -ne 0 ]]; then
|
||||
echo -e "\033[0;31mCommit blocked for security reasons.\033[0m\n"
|
||||
exit 1
|
||||
printf "\n\033[0;31mCommit blocked: security policy violation.\033[0m\n\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exit 0
|
||||
|
|
|
|||
|
|
@ -11,7 +11,8 @@ private: false
|
|||
|
||||
**[AGENT INSTRUCTION]**
|
||||
This is the primary navigation file. Read it first on every session before accessing individual pages.
|
||||
Maintain strict alphabetical sorting within each section.
|
||||
Append new entries at the bottom of the relevant section — do not reorder or rewrite sections.
|
||||
Alphabetical sorting is handled automatically by the pre-commit hook.
|
||||
Update `last_updated` in the YAML frontmatter on every edit.
|
||||
Entry format: `- [[folder/slug]] — One-line summary. \`maturity: <value>\``
|
||||
|
||||
|
|
|
|||
|
|
@ -9,6 +9,10 @@ private: false
|
|||
|
||||
# Operations Log: {{GENOME_NAME}}
|
||||
|
||||
**[ORCHESTRATOR]**
|
||||
Inject only the last 20 entries into agent context: `tail -n 20 wiki/log.md`
|
||||
The agent must never load or read the full log file — it grows unbounded.
|
||||
|
||||
**[AGENT INSTRUCTION]**
|
||||
This is an append-only system ledger. Never edit or delete previous entries.
|
||||
Append new entries at the bottom using the format defined below.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue