Lily Ge

~ se @ uwaterloo · nlp & ir ~

Hello! I'm Lily Ge; I also publish under my given name, Yijun Ge. I'm an undergraduate Software Engineering student at the University of Waterloo. My interests lie in Natural Language Processing, Information Retrieval, Agentic Systems, and Robots. I am fortunate to be advised by Professor Jimmy Lin.

Experiences

Machine Learning Engineering Intern

Grand Charter · Sep 2026 - Present · New York, NY

I rebuilt the integrated agent of the Presolve case management system, created agentic plaintiff intake processes for DearLegal, and developed voice agents to simulate clients for the outreach team to practice calls.

AI Engineer

Yupp AI · Aug 2025 - Mar 2026 · San Francisco, CA

I built RAG pipelines, enabled citation extraction across various model providers, and prepared data for annotation.

Publications

Neng Li*, Yijun Ge*, Xueguang Ma, and Jimmy Lin. 1ShotBench: Demonstrating One-Shot Prompt Reproducibility for Autonomous Coding Agents. Proceedings of the 35th International Conference on Information and Knowledge Management (CIKM 2026), pages 9149-9153, November 2026, Rome, Italy. Paper Code Read abstractHide abstract

The growing use of LLM agents raises a reproducibility problem: given the same task specification, different agents may follow different workflows, consume different resources, and produce varying artifacts and outcomes. We introduce the concept of One-Shot Prompt Reproducibility (1SPR): the consistency of one-shot execution across different agents given a fixed specification, without iterative human guidance. To operationalize this, we present 1ShotBench, a toolkit that repeatedly executes the same product requirement document (PRD) across isolated agent workspaces, capturing full traces, artifacts, and metadata, along with an LLM judge to evaluate results. We demonstrate a case study using software engineering tasks built around the Anserini information retrieval toolkit, though our system itself supports arbitrary tasks. 1ShotBench allows users to compare runs side by side, inspect how execution and implementation paths diverge despite a shared specification, and analyze how workflow differences relate to cost, latency, and correctness. Our central contribution is a reproducibility-oriented lens for examining agentic software development.

Sahel Sharifymoghaddam*, Lingwei Gu*, Yijun Ge, and Jimmy Lin. CMASS: A ClimbMix Agentic Search Suite for Evaluating Search Agents Across Heterogeneous Tasks. arXiv:2608.20317, October 2026. Paper Resources Read abstractHide abstract

Evaluating advances in agentic search requires testing whether methods generalize across tasks and corpus scale. Changing web search services impede controlled comparisons, while small corpora assembled around benchmark questions provide a limited test of generalization.

We introduce CMASS, the ClimbMix Agentic Search Suite, which projects three search benchmarks—BrowseComp-Plus, Total Recall QA, and DeepSearchQA—and the multi-hop question-answering dataset HotpotQA onto a shared, fixed corpus. The target, ClimbMix, is a pre-training corpus of 553M documents and approximately 400B tokens, assembled independently of the benchmark questions.

Our projection pipeline verifies grounding, answer uniqueness, and set completeness through independent agent audits, controlled revision, and human review. The 423 questions span entity identification, multi-hop reasoning, exhaustive answer sets, and aggregation. Evaluations of three models expose substantial headroom for open-weight agents and show that high answer accuracy can coexist with incomplete evidence coverage.

By enabling measurement of accuracy alongside evidence coverage, search effort, and closed-book performance, CMASS supports developing agents that find sufficient evidence and produce complete answers efficiently. The pipeline code and dataset are available at https://cmass.tech/.

Tz-Huan Hsu, Jheng-Hong Yang, Yijun Ge, and Jimmy Lin. Rethinking Agentic Search with Piika: Is Lexical Retrieval Sufficient? Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026), pages 1534–1550, October 2026, Budapest, Hungary. Paper Code Read abstractHide abstract

Does a lexical retriever suffice as large language models (LLMs) become more capable within an agentic search loop? We revisit this question by pairing BM25 with increasingly capable LLMs and introduce Piika, a search agent equipped with three tools for retrieving, browsing, and reading documents. On the BrowseComp-Plus benchmark, under the same GPT-5 + BM25 setup, Piika improves the accuracy from 58.3% to 74.6% while reducing cost from $400 to $95.

Despite relying only on lexical retrieval, it also outperforms the previously reported GPT-5 + dense-retriever setup, which achieves 73.0% accuracy at $361. Piika further scales with stronger LLMs, reaching 83.1% answer accuracy and 94.7% surfaced-evidence recall with GPT-5.5. Across model families, our results show that simple lexical retrieval, coupled with capable LLMs, a lightweight harness, and an effective tool interface, can remain highly competitive with more complex retrieval systems.

Source code and evaluation artifacts are available at https://piika.ai.

Yijun Ge, Zibo Guo, Sahel Sharifymoghaddam, and Jimmy Lin. MCP Servers for Pyserini and RankLLM: Enabling Agentic Retrieval-Augmented Generation. Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026), pages 5156-5160, July 2026, Melbourne, Australia. Paper Code (Pyserini) Code (RankLLM) Read abstractHide abstract

Large language models (LLMs) increasingly rely on retrieval-augmented generation (RAG), where search results are provided as external context to improve grounding and factuality. The Model Context Protocol (MCP) is an emerging industry standard for connecting LLMs to external tools. We present MCP servers for Pyserini and RankLLM, two widely used information retrieval research toolkits, enabling their retrieval and reranking capabilities to be easily accessed by LLM agents. Our system exposes sparse and dense retrieval over prebuilt indexes, multimodal search, LLM-based reranking, and evaluation based on relevance judgments, all as tools callable by LLMs. This integration supports rapid experimentation with agentic RAG workflows and facilitates comparisons across models and clients. We demonstrate end-to-end retrieval, reranking, RAG, and multimodal search using both open-source and proprietary LLMs, bridging IR research infrastructure with LLM tool ecosystems.

Sahel Sharifymoghaddam*, Yijun Ge*, Raghav Vasudeva, and Jimmy Lin. Lighting the Way for BRIGHT: Reproducible Baselines with Anserini, Pyserini, and RankLLM. Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026), pages 3059-3069, July 2026, Melbourne, Australia. Paper Code (Anserini) Code (Pyserini) Read abstractHide abstract

Retrieval benchmarks for large language models (LLMs) should reflect the long, reasoning-intensive queries typical of retrieval-augmented generation (RAG). We present a systematic study of BRIGHT, a reasoning-focused retrieval benchmark, along with strong, reproducible reference methods integrated into Anserini, Pyserini, and RankLLM. We evaluate lexical, sparse, dense, and fusion-based retrievers, as well as LLM rerankers, under long-query settings.

In reproducing BRIGHT's lexical baseline, we identify a key under-documented detail: query-side BM25 (BM25Q), which applies BM25 weighting to the query itself. On long, multi-sentence queries, BM25Q consistently outperforms standard BM25, making it the strongest lexical baseline for reasoning-oriented retrieval. We further audit the BRIGHT corpus, uncovering data quality issues that impact evaluation, and offer mitigation.

Finally, we study the generalizability of BM25Q across five additional benchmarks, finding its gains largely specific to BRIGHT, while fusion with standard BM25 provides the most consistent improvements across datasets.

Yijun Ge*, Kushaldeep Mujral*, Karthik Nambiar, and Jimmy Lin. rosaOS: Agentic Operating System for Embodied LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 473-480, July 2026, San Diego, California. Paper Code Demo Read abstractHide abstract

We present rosaOS, an open-source agentic operating system for embodied LLMs: interactive, LLM-driven agents coordinate various software tools and physical devices through a desktop companion, the Reachy Mini robot. Existing LLM–robotic systems are generally built as a tight, intertwined stack, making it difficult to switch hardware, add extra capabilities, or expand to multiple devices without bespoke integration.

Our system aims to provide a classic OS-inspired architecture where an agentic kernel manages all task execution and mediates device access, while process agents invoke tools to perform actions. We adopt industry-standard interfaces with MCP for agentic tool-calling and ROS for robot interactions, and demonstrate rosaOS on a multi-device setup including a quadruped robot, a wheeled mobile robot, and a smart lamp, all controlled through interactions with the Reachy Mini.

By incorporating MCP extensibility with ROS hardware interoperability, rosaOS enables a plug-and-play ecosystem for building embodied agentic systems. Our OS is available at rosaos.ai.

Yijun Ge, Zijian Chen, and Jimmy Lin. QuackIR: Retrieval in DuckDB and Other Relational Database Management Systems. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 492-500, November 2025, Suzhou, China. Paper Code Read abstractHide abstract

Enterprises today are increasingly compelled to adopt dedicated vector databases for retrieval-augmented generation (RAG) in applications based on large language models (LLMs). As a potential alternative for these vector databases, we propose that organizations leverage existing relational databases for retrieval, which many have already deployed in their enterprise data lakes, thus minimizing additional complexity in their software stacks. To demonstrate the simplicity and feasibility of this approach, we present QuackIR, an information retrieval (IR) toolkit built on relational database management systems (RDBMSes), with integrations in DuckDB, SQLite, and PostgreSQL.

Using QuackIR, we benchmark the sparse and dense retrieval capabilities of these popular RDBMSes and demonstrate that their effectiveness is comparable to baselines from established IR toolkits. Our results highlight the potential of relational databases as a simple option for RAG scenarios due to their established widespread usage and the easy integration of retrieval abilities. Our implementation is available at quackir.io.

Projects

Ruth's Super Amazing AI Adventure!

Python · LangChain · Pinecone · SQLite · Groq · Flask · Diffusers

An AI-driven visual novel with LLM dialogue, RAG-backed long-term memory, local stable diffusion, and custom game logic through a Flask backend.

AccessAIbility

GeeseHacks 2025 Award Winner· Devpost · Python · MediaPipe · OpenCV

Webcam-based keyboard input from ASL hand gestures and MediaPipe gesture shortcuts for control keys, with modular AI components and real-time OpenCV video.

Misc