diff --git a/.claude/commands/wiki-graph.md b/.claude/commands/wiki-graph.md new file mode 100644 index 0000000..64c0338 --- /dev/null +++ b/.claude/commands/wiki-graph.md @@ -0,0 +1,18 @@ +Build the LLM Wiki knowledge graph. + +Usage: /wiki-graph + +First try running: python tools/build_graph.py --open + +If that fails (missing dependencies), build the graph manually: + +1. Use Grep to find all [[wikilinks]] across every file in wiki/ +2. Build a nodes list: one node per wiki page, with id=relative-path, label=title, type from frontmatter +3. Build an edges list: one edge per [[wikilink]], tagged EXTRACTED +4. Infer additional implicit relationships between pages not captured by wikilinks — tag these INFERRED with a confidence score (0.0–1.0); tag low-confidence ones AMBIGUOUS +5. Write graph/graph.json with {nodes, edges, built: today} +6. Write graph/graph.html as a self-contained vis.js page (nodes colored by type, edges colored by type, interactive, searchable) + +After building, summarize: node count, edge count, breakdown by type, and the most connected nodes (hubs). + +Append to wiki/log.md: ## [today's date] graph | Knowledge graph rebuilt diff --git a/.claude/commands/wiki-ingest.md b/.claude/commands/wiki-ingest.md new file mode 100644 index 0000000..f8e3a37 --- /dev/null +++ b/.claude/commands/wiki-ingest.md @@ -0,0 +1,18 @@ +Ingest a source document into the LLM Wiki. + +Usage: /wiki-ingest $ARGUMENTS + +$ARGUMENTS should be the path to a file in raw/, e.g. `raw/articles/my-article.md` + +Follow the Ingest Workflow defined in CLAUDE.md exactly: +1. Read the source file at the given path +2. Read wiki/index.md and wiki/overview.md for current context +3. Write wiki/sources/.md (source page format per CLAUDE.md) +4. Update wiki/index.md — add the new entry under Sources +5. Update wiki/overview.md — revise synthesis if warranted +6. Create/update entity pages (wiki/entities/) for key people, companies, projects +7. Create/update concept pages (wiki/concepts/) for key ideas and frameworks +8. Flag any contradictions with existing wiki content +9. Append to wiki/log.md: ## [today's date] ingest | + +After completing all writes, summarize: what was added, which pages were created or updated, and any contradictions found. diff --git a/.claude/commands/wiki-lint.md b/.claude/commands/wiki-lint.md new file mode 100644 index 0000000..fe45b69 --- /dev/null +++ b/.claude/commands/wiki-lint.md @@ -0,0 +1,19 @@ +Health-check the LLM Wiki for issues. + +Usage: /wiki-lint + +Follow the Lint Workflow defined in CLAUDE.md: + +Structural checks (use Grep and Glob tools): +1. Orphan pages — wiki pages with no inbound [[wikilinks]] from other pages +2. Broken links — [[WikiLinks]] pointing to pages that don't exist +3. Missing entity pages — names referenced in 3+ pages but lacking their own page + +Semantic checks (read and reason over page content): +4. Contradictions — claims that conflict between pages +5. Stale summaries — pages not updated after newer sources changed the picture +6. Data gaps — important questions the wiki can't answer; suggest specific sources to find + +Output a structured markdown lint report. At the end, ask if the user wants it saved to wiki/lint-report.md. + +Append to wiki/log.md: ## [today's date] lint | Wiki health check diff --git a/.claude/commands/wiki-query.md b/.claude/commands/wiki-query.md new file mode 100644 index 0000000..8a6a6df --- /dev/null +++ b/.claude/commands/wiki-query.md @@ -0,0 +1,14 @@ +Query the LLM Wiki and synthesize an answer. + +Usage: /wiki-query $ARGUMENTS + +$ARGUMENTS is the question to answer, e.g. `What are the main themes across all sources?` + +Follow the Query Workflow defined in CLAUDE.md: +1. Read wiki/index.md to identify the most relevant pages +2. Read those pages (up to ~10 most relevant) +3. Synthesize a thorough markdown answer with [[PageName]] wikilink citations +4. Include a ## Sources section at the end listing pages you drew from +5. Ask the user if they want the answer saved as wiki/syntheses/<slug>.md + +If the wiki is empty, say so and suggest running /wiki-ingest first. diff --git a/CLAUDE.md b/CLAUDE.md index ef54216..a57d203 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,25 @@ # LLM Wiki Agent — Schema & Workflow Instructions -This document defines how Claude maintains this wiki. Follow these conventions exactly in every session. +This wiki is maintained entirely by Claude Code. No API key or Python scripts needed — just open this repo in Claude Code and talk to it. + +## Slash Commands (Claude Code) + +| Command | What to say | +|---|---| +| `/wiki-ingest` | `ingest raw/my-article.md` | +| `/wiki-query` | `query: what are the main themes?` | +| `/wiki-lint` | `lint the wiki` | +| `/wiki-graph` | `build the knowledge graph` | + +Or just describe what you want in plain English: +- *"Ingest this file: raw/papers/attention-is-all-you-need.md"* +- *"What does the wiki say about transformer models?"* +- *"Check the wiki for orphan pages and contradictions"* +- *"Build the graph and show me what's connected to RAG"* + +Claude Code reads this file automatically and follows the workflows below. + +--- ## Directory Layout @@ -13,10 +32,13 @@ wiki/ # Claude owns this layer entirely sources/ # One summary page per source document entities/ # People, companies, projects, products concepts/ # Ideas, frameworks, methods, theories -graph/ # Auto-generated graph data — regenerate with build_graph.py -tools/ # CLI scripts + syntheses/ # Saved query answers +graph/ # Auto-generated graph data +tools/ # Optional standalone Python scripts (require ANTHROPIC_API_KEY) ``` +--- + ## Page Format Every wiki page uses this frontmatter: @@ -31,23 +53,24 @@ last_updated: YYYY-MM-DD --- ``` -Use `[[PageName]]` wikilinks to link to other wiki pages. These are parsed by build_graph.py. +Use `[[PageName]]` wikilinks to link to other wiki pages. --- ## Ingest Workflow -Triggered when user runs: `python tools/ingest.py <source-path>` +Triggered by: *"ingest <file>"* or `/wiki-ingest` Steps (in order): -1. Read the source document fully -2. Write `wiki/sources/<slug>.md` — title, date, key claims, key quotes, tags, links to entity/concept pages -3. Update `wiki/index.md` — add entry under the correct section -4. Update `wiki/overview.md` — revise synthesis if the source adds new perspectives, themes, or contradicts prior conclusions -5. Update existing entity pages that this source mentions; create new entity pages if needed -6. Update existing concept pages that this source discusses; create new concept pages if needed -7. Flag any contradictions with existing wiki content in the log entry -8. Append to `wiki/log.md` with this prefix format: `## [YYYY-MM-DD] ingest | <Title>` +1. Read the source document fully using the Read tool +2. Read `wiki/index.md` and `wiki/overview.md` for current wiki context +3. Write `wiki/sources/<slug>.md` — use the source page format below +4. Update `wiki/index.md` — add entry under Sources section +5. Update `wiki/overview.md` — revise synthesis if warranted +6. Update/create entity pages for key people, companies, projects mentioned +7. Update/create concept pages for key ideas and frameworks discussed +8. Flag any contradictions with existing wiki content +9. Append to `wiki/log.md`: `## [YYYY-MM-DD] ingest | <Title>` ### Source Page Format @@ -82,48 +105,53 @@ source_file: raw/... ## Query Workflow -Triggered when user runs: `python tools/query.py "<question>"` +Triggered by: *"query: <question>"* or `/wiki-query` Steps: 1. Read `wiki/index.md` to identify relevant pages -2. Read the relevant pages -3. Synthesize an answer with inline citations as wikilinks: `[[PageName]]` -4. Ask the user if they want the answer filed as a new synthesis page in `wiki/` +2. Read those pages with the Read tool +3. Synthesize an answer with inline citations as `[[PageName]]` wikilinks +4. Ask the user if they want the answer filed as `wiki/syntheses/<slug>.md` --- ## Lint Workflow -Triggered when user runs: `python tools/lint.py` +Triggered by: *"lint the wiki"* or `/wiki-lint` -Check for: +Use Grep and Read tools to check for: - **Orphan pages** — wiki pages with no inbound `[[links]]` from other pages -- **Contradictions** — claims that conflict across pages -- **Stale summaries** — pages not updated after newer sources changed the picture -- **Missing entity pages** — entities mentioned in 3+ pages but lacking their own page - **Broken links** — `[[WikiLinks]]` pointing to pages that don't exist -- **Data gaps** — important questions the wiki cannot answer — suggest new sources to find +- **Contradictions** — claims that conflict across pages +- **Stale summaries** — pages not updated after newer sources +- **Missing entity pages** — entities mentioned in 3+ pages but lacking their own page +- **Data gaps** — questions the wiki can't answer; suggest new sources -Output a lint report as markdown. +Output a lint report and ask if the user wants it saved to `wiki/lint-report.md`. --- ## Graph Workflow -Triggered when user runs: `python tools/build_graph.py` +Triggered by: *"build the knowledge graph"* or `/wiki-graph` -- Pass 1: Parse all `[[wikilinks]]` in wiki pages → deterministic edges tagged `EXTRACTED` -- Pass 2: Call Claude API to infer implicit relationships not captured by wikilinks → edges tagged `INFERRED` with confidence score -- Tag ambiguous relationships as `AMBIGUOUS` -- Use Louvain community detection to cluster nodes -- Output `graph/graph.json` and `graph/graph.html` -- Cache by SHA256 of page content — only reprocess changed pages +When the user asks to build the graph, run `tools/build_graph.py` which: +- Pass 1: Parses all `[[wikilinks]]` → deterministic `EXTRACTED` edges +- Pass 2: Infers implicit relationships → `INFERRED` edges with confidence scores +- Runs Louvain community detection +- Outputs `graph/graph.json` + `graph/graph.html` + +If the user doesn't have Python/dependencies set up, instead generate the graph data manually: +1. Use Grep to find all `[[wikilinks]]` across wiki pages +2. Build a node/edge list +3. Write `graph/graph.json` directly +4. Write `graph/graph.html` using the vis.js template --- ## Naming Conventions -- Source slugs: `kebab-case` matching the source filename +- Source slugs: `kebab-case` matching source filename - Entity pages: `TitleCase.md` (e.g. `OpenAI.md`, `SamAltman.md`) - Concept pages: `TitleCase.md` (e.g. `ReinforcementLearning.md`, `RAG.md`) - Source pages: `kebab-case.md` @@ -151,7 +179,7 @@ Triggered when user runs: `python tools/build_graph.py` ## Log Format -Each entry starts with `## [YYYY-MM-DD] <operation> | <title>` so it's parseable: +Each entry starts with `## [YYYY-MM-DD] <operation> | <title>` so it's grep-parseable: ``` grep "^## \[" wiki/log.md | tail -10 diff --git a/README.md b/README.md index e1eaaec..0f74f85 100644 --- a/README.md +++ b/README.md @@ -2,60 +2,60 @@ [![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) -**A personal knowledge base that builds and maintains itself.** Drop in source documents — articles, papers, notes — and the LLM reads them, extracts the knowledge, and integrates everything into a persistent, interlinked wiki. You never write the wiki. The LLM does. +**A personal knowledge base that builds and maintains itself.** Drop in source documents — articles, papers, notes — and the LLM reads them, extracts the knowledge, and integrates everything into a persistent, interlinked wiki. You never write the wiki. Claude does. Unlike RAG systems that re-derive knowledge from scratch on every query, LLM Wiki Agent compiles knowledge once and keeps it current. Cross-references are pre-built. Contradictions are flagged at ingest time. Every new source makes the wiki richer. ## How It Works ``` -You drop a source → LLM reads it → wiki pages are created/updated → graph is rebuilt +You drop a source → Claude reads it → wiki pages are created/updated → graph is rebuilt -You ask a question → LLM reads relevant wiki pages → synthesizes answer with citations +You ask a question → Claude reads relevant wiki pages → synthesizes answer with citations ``` Three layers: - **`raw/`** — your source documents (immutable, you own this) -- **`wiki/`** — LLM-maintained markdown pages (Claude writes, you read) +- **`wiki/`** — Claude-maintained markdown pages (Claude writes, you read) - **`graph/`** — auto-generated knowledge graph visualization -## Quick Start +## Quick Start — Claude Code (no API key needed) + +Open this repo in [Claude Code](https://claude.ai/code): ```bash git clone https://github.com/SamurAIGPT/GPT-Agent.git cd GPT-Agent +claude # opens Claude Code in this repo +``` + +Claude Code reads `CLAUDE.md` automatically. Then just talk to it: + +``` +# Drop a source into raw/ first, then: +/wiki-ingest raw/articles/my-article.md + +/wiki-query what are the main themes across all sources? + +/wiki-lint + +/wiki-graph +``` + +Or in plain English: *"Ingest this paper"*, *"What does the wiki say about X?"*, *"Check for contradictions"* + +## Quick Start — Standalone Python (requires API key) + +```bash pip install -r requirements.txt export ANTHROPIC_API_KEY=your_key_here -``` -Add your first source: - -```bash -# Drop a source document into raw/ -cp my-article.md raw/articles/my-article.md - -# Ingest it — LLM reads, extracts, and files knowledge into the wiki python tools/ingest.py raw/articles/my-article.md -``` - -Query the wiki: - -```bash -python tools/query.py "What are the main themes across all sources?" -python tools/query.py "How does X relate to Y?" --save # save answer back to wiki -``` - -Build the knowledge graph: - -```bash -python tools/build_graph.py --open # opens graph.html in browser -``` - -Health-check the wiki: - -```bash -python tools/lint.py --save # checks for orphans, contradictions, gaps +python tools/query.py "What are the main themes?" +python tools/query.py "How does X relate to Y?" --save +python tools/build_graph.py --open +python tools/lint.py --save ``` ## Architecture @@ -81,16 +81,29 @@ tools/ CLAUDE.md ← schema and workflow instructions for the LLM ``` -## Tools +## Commands + +### Claude Code (primary — no API key) + +| Slash command | What it does | +|---|---| +| `/wiki-ingest <file>` | Read a source, update wiki pages, append to log | +| `/wiki-query <question>` | Search wiki, synthesize answer with citations | +| `/wiki-lint` | Check for orphans, broken links, contradictions, gaps | +| `/wiki-graph` | Build knowledge graph (`graph.json` + `graph.html`) | + +Or describe what you want in plain English — Claude Code follows `CLAUDE.md` and does the right thing. + +### Standalone Python (optional — requires `ANTHROPIC_API_KEY`) | Command | What it does | |---|---| -| `python tools/ingest.py <file>` | Read a source, update wiki pages, append to log | -| `python tools/query.py "<question>"` | Search wiki, synthesize answer with citations | -| `python tools/query.py "<question>" --save` | Same, and file the answer back as a wiki page | -| `python tools/lint.py` | Check for orphans, broken links, contradictions, gaps | -| `python tools/build_graph.py` | Build `graph.json` + `graph.html` from wiki | -| `python tools/build_graph.py --no-infer` | Build graph without semantic inference (faster) | +| `python tools/ingest.py <file>` | Ingest a source | +| `python tools/query.py "<question>"` | Query the wiki | +| `python tools/query.py "<question>" --save` | Query and file answer back | +| `python tools/lint.py` | Lint the wiki | +| `python tools/build_graph.py` | Build graph | +| `python tools/build_graph.py --no-infer` | Build graph (skip inference, faster) | | `python tools/build_graph.py --open` | Build and open in browser | ## The Graph