The Contradiction Between Surging Agent Requests and Uncontrolled Costs
Over the past six months, invocation volume for third-party AI assistants in the Lark ecosystem has surged 9.4-fold. Logically, the end-of-month token bill should leave teams gasping—but in reality, it rose by just 3%. This isn’t because we secured some magical discount; it’s because we completely redesigned our underlying engineering architecture from the ground up.
Today, using intelligent work Agents to process Lark messages, draft documents, and run approvals has become second nature. Business teams take it for granted—asking AI office tools to effortlessly summarize meetings or analyze multi-dimensional spreadsheet data. Yet IT teams staring at their monitoring dashboards see only an exponential spike in interaction frequency, plunging them headlong into a compute black hole. On one side: business demanding value. On the other: finance enforcing budget constraints. That cost red line hangs, immovable, over engineering teams’ heads.
When bills exceed targets, many teams instinctively reach for two levers: rate limiting or model downgrading—slashing invocation quotas or swapping large models for cheaper, smaller ones. This is classic “throwing the baby out with the bathwater.” Rate limiting shackles business innovation; downgrading is even worse—hallucinations run rampant, and intelligent agents regress into rigid, rule-based engines. We refuse to obsess over “how to lower unit price.” Instead, we focus on one mission: *eliminating waste*.
Put plainly, AI spending must be treated as a measurable, decomposable engineering problem—one solved through system-level optimization to squeeze every drop of valueless token “water” out of the pipeline. To achieve explosive growth in invocation volume without rising costs, we must first understand the mathematical logic hidden behind the bill. Next, we’ll dissect and unpack the cost formula for AI-powered office conversations.
Deconstructing the Cost Formula for AI-Powered Office Conversations
To eliminate waste, we must first quantify it precisely. We’ve broken down Lark’s formerly opaque “monthly AI bill” into a five-factor multiplicative model:
Total Cost = Number of Active Users × Average Conversations per User × Tokens Consumed per Turn × Average Interaction Turns per Conversation × Model Unit Price
With this formula, the “black box” viewed by finance becomes a transparent “white box” for engineers. Consider the first two factors: number of active users and average conversations per user. These are fundamentally business-value metrics. When employees routinely summon intelligent work Agents in Lark chats, documents, and calendars, it signals that AI has genuinely embedded itself into daily workflows. These metrics shouldn’t be capped—they should be actively encouraged to grow. They represent real, tangible business growth.
The true cost black holes lie in the last three factors: tokens consumed per turn, average interaction turns per conversation, and model unit price.
Model unit price is market-determined—we have limited room to negotiate. But context size per turn and interaction turn count? Those are entirely under engineering’s control. Many teams integrating AI office tools default to dumping all business data wholesale into prompts—or letting models spin endlessly in “trial-and-error–correction” loops. *That’s* where runaway costs originate.
Once the math is clear, the optimization target becomes obvious: allow the first two business metrics to soar freely, while aggressively squeezing the last three systemic cost drivers via architectural intervention. What once felt like an intractable billing anxiety thus transforms into concrete, actionable lines of code optimization.
Compressing Request Payload Size and Context Bloat
Having deconstructed the formula, our first optimization naturally targets the most volatile variable: tokens consumed per turn.
When integrating with Lark Docs or multi-dimensional spreadsheets, the laziest—and most common—engineering anti-pattern is “full-data injection”: blindly stuffing entire documents or full-table datasets into prompts. Not only does this send the billing baseline soaring, but it also dilutes the model’s attention until it’s utterly unfocused. Context engineering isn’t about who ingests more data—it’s about who achieves the highest signal-to-noise ratio.
So we implemented dynamic trimming and on-demand retrieval. When an intelligent work Agent reads a Lark Doc, the system no longer naively sends the full text. Instead, leveraging external vector search services alongside Lark’s open document and multi-dimensional spreadsheet APIs, it retrieves only highly relevant text snippets on demand. For multi-dimensional spreadsheets, we sync only the table schema and representative samples—not the full dataset. This “on-demand feeding” strategy slashed the average token payload size per request by 68%.
Historical context bloat in multi-turn dialogues is another hidden black hole. The deeper the conversation goes, the more earlier redundant instructions consume precious token budget. To address this, we introduced a dual mechanism: a sliding window plus structured summarization. Recent turns retain raw text; older history is offloaded to a lightweight model and compressed into a “fact state machine”—retaining only key entities, current intent, and confirmed parameters, while ruthlessly pruning repetitive phrasing and procedural filler.
Additionally, at the pre-API-call service layer, we implemented intent recognition and result caching. For high-frequency semantically equivalent requests (e.g., “check this week’s calendar”), we reuse previously computed results—bypassing redundant API calls and model inference entirely.
Squeezing valueless token “water” means pushing information density to its physical limit. But once per-request payload size hits that ceiling, the next hard nut to crack is reducing idle model “spinning” across complex business logic.
Optimizing Interaction Turns and Business Logic Encapsulation
Once intelligent work Agents dive deep into core business operations, compressing per-request payload alone can no longer contain costs. The real billing black hole lies in the “thinking” and “trial-and-error” cycles embedded within multi-turn interactions.
Take scheduling a cross-departmental video meeting. If left to the model alone, the Agent must first query Lark Calendar for availability, then verify participants against the org chart, and only then initiate meeting creation. This chain—“plan → call → observe → re-plan”—easily spawns 3–5 interaction turns. Each burns tokens—and parameter-parsing errors frequently trigger retries. This isn’t intelligence; it’s compute spinning in place.
Our principle is clear: *never force large models to do workflow orchestration grunt work*. So we built a custom service layer that integrates directly with Lark’s open APIs—aggregating cross-module operations (e.g., calendar lookup + participant validation + meeting creation) into single, atomic calls, eliminating multi-turn polling by the Agent.
How? We abstracted the cross-departmental meeting scenario described above into a high-level tool. The Agent outputs its intent just once; the gateway then executes calendar queries, participant verification, and meeting creation *in parallel* at the infrastructure layer—and returns only the consolidated result to the model. This engineering encapsulation slashed average interaction turns for complex scenarios from 4.2 down to just 1, cutting inference overhead by 68%.
Replace model polling with engineering encapsulation. Replace probabilistic reasoning with deterministic code. The ultimate rule for lowering AI office tool operating costs has never been “make the model smarter”—but rather, architecturally *remove its opportunity to err*.
Introducing Business-Fact Anchors and Global Architecture Governance
Once interaction turns are minimized via engineering encapsulation, residual token waste usually stems from models “feeling their way blindly” through business context. To solve this, we built a structured business-context index inside our proprietary system—synchronizing Lark Docs and multi-dimensional spreadsheet data via Lark’s open APIs—to deliver precise, fact-based context slices to third-party AI assistants.
Traditional Retrieval-Augmented Generation (RAG) often triggers repeated retries due to imprecise recall. We instead leveraged Lark’s open capabilities to integrate with enterprise knowledge bases—consolidating scattered approval records, project timelines, and meeting resolutions into a structured context graph. When Agents perform tasks, they fetch precise fact slices directly from this graph—no more blind searching across massive unstructured datasets. This architectural shift eliminated *all* ineffective retrieval attempts per request, and cut retry overhead caused by hallucinations by 81%.
Step back and survey this entire optimization chain: extreme compression of context volume, engineering encapsulation of interaction turns, and global anchoring of business facts. This isn’t a collection of ad-hoc patches—it’s a portable, battle-tested methodology. We broke down the AI office tool operating bill into five multiplicative cost factors—and surgically optimized each one.
Built atop Lark’s open ecosystem foundation, both internally developed applications and third-party AI assistants can plug into this governance architecture via standardized interfaces. A 9.4-fold surge in invocation volume with near-flat total cost wasn’t luck—or simply cheaper compute. It was achieved by relentlessly tracking the flow of *every single token*.
As workflows evolve from interactive assistance toward fully managed intelligent agents, the anxiety around runaway costs will inevitably fade. This challenge was never some inscrutable algorithmic black box. It’s a measurable, decomposable, solvable engineering problem. Squeeze out every wasteful token. Turn compute into genuine productivity. That’s our engineering answer to the AI era.