wehub-resource-sync
26382a7ac6
CI / Clippy (push) Failing after 15m13s
CI / Test (ubuntu-latest) (push) Failing after 16m1s
CI / Test (macos-latest) (push) Has been cancelled
CI / Test (windows-latest) (push) Has been cancelled
CI / Build (no embeddings / no ORT) (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Cookbook (Node) (push) Has been cancelled
CI / Pi Extension (Node) (push) Has been cancelled
CI / Rust SDK (lean-ctx-client) (push) Has been cancelled
CI / Embed SDK (lean-ctx-sdk) (push) Has been cancelled
CI / Python SDK (leanctx) (push) Has been cancelled
CI / Hermes Plugin (Python) (push) Has been cancelled
CI / SDK Conformance Matrix (push) Has been cancelled
CI / Coverage (push) Has been cancelled
CI / cargo-deny (push) Has been cancelled
CI / Adversarial Safety (push) Has been cancelled
CI / Benchmarks (push) Has been cancelled
CI / Output-Quality Gate (eval A/B) (push) Has been cancelled
CI / Documentation (push) Has been cancelled
CI / CI Green (push) Has been cancelled
JetBrains Plugin / Actionlint (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (rust) (push) Has been cancelled
JetBrains Plugin / Validation (push) Has been cancelled
JetBrains Plugin / Build (push) Has been cancelled
JetBrains Plugin / Test (push) Has been cancelled
Security Check / Security Scan (push) Has been cancelled
8 lines
3.0 KiB
Plaintext
8 lines
3.0 KiB
Plaintext
# lean-ctx LoCoMo reference suite — NDJSON, one sample per line.
|
|
# Real, verifiable facts only; every gold answer is grounded in a conversation turn
|
|
# AND objectively true. Run the official LoCoMo dataset via `--suite <path>`.
|
|
# Schema: { id, sessions:[{session_id, turns:[{speaker,text}]}], qa:[{question, answers:[], category}] }
|
|
# Categories: 1=single-hop, 2=multi-hop, 3=temporal, 4=open-domain, 5=adversarial.
|
|
{"id":"trip-japan","sessions":[{"session_id":"s1","turns":[{"speaker":"Alex","text":"I'm planning a trip to Japan next spring."},{"speaker":"Sam","text":"Nice. Tokyo is the capital of Japan and one of the largest cities in the world."},{"speaker":"Alex","text":"I really want to see Mount Fuji. It is the highest mountain in Japan at 3,776 meters."},{"speaker":"Sam","text":"You should also visit Kyoto. It was the capital of Japan before Tokyo, until the year 1868."}]},{"session_id":"s2","turns":[{"speaker":"Alex","text":"I booked my flights. I am leaving on March 15th."},{"speaker":"Sam","text":"The cherry blossoms in Tokyo usually peak in late March, so your timing is good."},{"speaker":"Alex","text":"For photos I will bring my Canon EOS R5 camera."}]}],"qa":[{"question":"What is the highest mountain in Japan?","answers":["Mount Fuji"],"category":1},{"question":"How tall is Mount Fuji?","answers":["3,776 meters","3776"],"category":1},{"question":"What was the capital of Japan before Tokyo?","answers":["Kyoto"],"category":1},{"question":"Until what year was Kyoto the capital of Japan?","answers":["1868"],"category":3},{"question":"On what date is Alex leaving for the trip?","answers":["March 15th","March 15"],"category":3},{"question":"Which camera will Alex bring for photos?","answers":["Canon EOS R5"],"category":1},{"question":"What is the capital of Japan?","answers":["Tokyo"],"category":1}]}
|
|
{"id":"study-group","sessions":[{"session_id":"s1","turns":[{"speaker":"Sam","text":"For the chemistry exam, remember that the chemical formula of water is H2O."},{"speaker":"Alex","text":"And the speed of light in a vacuum is about 299,792 kilometers per second."},{"speaker":"Sam","text":"For biology, the adult human body has 206 bones."}]},{"session_id":"s2","turns":[{"speaker":"Alex","text":"In geography we learned that the Pacific is the largest ocean on Earth."},{"speaker":"Sam","text":"Right, and the Sahara is the largest hot desert in the world."},{"speaker":"Alex","text":"Also, the Nile is generally regarded as the longest river in the world."}]}],"qa":[{"question":"What is the chemical formula of water?","answers":["H2O"],"category":1},{"question":"How many bones are in the adult human body?","answers":["206"],"category":1},{"question":"What is the largest ocean on Earth?","answers":["Pacific"],"category":1},{"question":"What is the largest hot desert in the world?","answers":["Sahara"],"category":1},{"question":"Which river is regarded as the longest in the world?","answers":["Nile"],"category":1},{"question":"What is the speed of light in a vacuum?","answers":["299,792 kilometers per second","299792"],"category":1}]}
|