Liquid <> .txt Collaboration | Blog | Liquid AI

In practical edge applications like smart home systems, industrial IoT sensors, or mobile assistants, we need models to generate function calls as accurately and quickly as possible. These devices face three critical challenges:

1. Limited processing power - The iPhone 15 Pro achieves 2.15 TFLOPS (FP32), which is 31 times less than a single H200 GPU
2. Latency requirements - Users expect near-instant responses (less than 300 ms)
3. Reliability demands - Function calls must work correctly the first time to meet the latency requirements.

Traditional approaches to function calling with LLMs often fail to meet these constraints. Models are too large, generation is too slow, and outputs can be inconsistent. This creates a significant barrier to deploying AI assistants that need to interact reliably with hardware and software systems at the edge.

The Solution: LFM2-350M + dotgrammar

LFM2-350M: AI That Fits on the Edge

LFM2-350M is an LLM developed by Liquid AI specifically for high-performance edge deployment with a custom lightweight architecture:

• Minimal footprint: It uses less than 1 GB of RAM without quantization, with minimal increase with long inputs
• Responsive performance: Delivers sub-100ms inference times on common edge hardware
• High quality: Achieves performance in terms of knowledge and reasoning on par with significantly larger models.

dotgrammar: Grammar-Based Generation

dotgrammar is a library developed by .txt for high-performance structured outputs using context-free grammars (CFGs). Complementing LFM2-350M's efficiency, dotgrammar ensures reliable function calling through:

• CFG constraints: It guarantees syntactically valid outputs every time
• Token-efficient function formats: Reducing generation time and improving responsiveness with Pythonic function calls.
• Zero runtime overhead: It enforces constraints without increasing inference latency

Let's explore how these technologies work together.

What is function calling?

Function calling is a technique where language models output structured function calls rather than natural language text. It enables AI assistants to interact with external tools, APIs, and hardware in a reliable, programmatic way.

For instance, if you want a model to control a media player, you need it to generate precise commands that your application can parse and execute:

User: "Play Spotify at high volume"
AI: play_media(device="speaker", source="spotify", volume=1)

This approach effectively makes LLMs speak your application's native language, bridging the gap between natural language understanding and actionable commands.

The traditional JSON approach

Most LLM providers implement function calling using JSON objects. While functional, this approach creates significant inefficiencies for edge deployment:

{
  "name": "play_media",
  "parameters": {
    "device": "speaker",
    "source": "spotify",
    "volume": 1
  }
}

This JSON representation requires 37 tokens to generate, where each token increases latency. Moreover, LLMs are not reliable when it comes to generating properly formatted nested structures, such as JSON.

The pythonic alternative

By contrast, a more compact Pythonic representation:

play_media(device="speaker", source="spotify", volume=1)

Requires just 14 tokens. It’s a 2.6x reduction that directly translates to faster generation times without sacrificing expressivity. Because LLMs are trained on a lot of code, it’s also a more reliable and natural approach to implement function calling.

The need for structured generation

Despite this training, LLMs can still make mistakes. These errors become especially problematic in edge applications where:

• Parameters have strict constraints (e.g., volume must be between 0-1)
• Parameter combinations must follow business logic (e.g., "speaker" and "netflix" is an invalid combination)
• Latency requirements leave no room for error handling and retries

For truly reliable function calls on the edge, we need structured generation that enforces output format in a deterministic way. This is where context-free grammars (CFGs) come in.

A context-free grammar (CFG) is a set of rules that defines exactly what combinations of characters or words are allowed, creating a "railroad track" that generation must follow.

Think of a CFG like a system for composing music with clear rules. Technically, a CFG consists of:

• Terminal symbols: The actual characters that appear in the final output
• Non-terminal symbols: Variables that get replaced according to rules
• Production rules: Instructions for replacing non-terminals with terminals or other non-terminals

For example, this grammar ensures correct play_media function calls:

Call ::= "play_media" "(" Arguments ")"
Arguments ::= "device=\"" Device "\"" ", " "source=\"" Source "\"" ", " "volume=" Volume
Device ::= "speaker" | "tv"
Source ::= "spotify" | "netflix" | "youtube"
Volume ::= "0" | "0.1" | "0.2" | ... | "1.0"

In this example:

• Terminal symbols include characters like "play_media", "(", and ")"
• Non-terminal symbols include Call, Arguments, Device, etc.
• The symbol "::=" means "can be replaced with"
• The symbol "|" means "OR" (indicating a choice)

Grammar-based structured generation provides the following guarantees:

• Security: Prevents injection attacks by strictly limiting what can be generated
• Reliability: Guarantees parseable output
• Validation: Enforces business rules at the generation stage

Demo: Smart Home Control

Let's see how LFM2-350M and dotgrammar work together in a practical edge application: a smart home assistant running locally on a hub device.

In this use case, the model will take in user queries and perform actions to set the state of various systems at home, like dimming lights, closing blinds, and playing media. Some functions like theater_mode() don’t require any arguments, but others like set_display() need more fine-grained control.

We will implement this using .txt’s dotgrammar library, which provides latency-free grammar-based structured generation.

Example 1: Simple function with unconstrained arguments

Let’s say a user just finished watching a movie, and they want to create a note about it. The smart home might have access to a save_note(str) function, which takes an arbitrary string as input.

In this case, we still want to constrain the output to ensure the function call is properly formatted, but we do not want to constrain the argument. This can be achieved in dotgrammar like this:

note_grammar = """
    ?start: "save_note(" UNESCAPED_STRING ")"
    %import common.UNESCAPED_STRING
"""

Example 2: One function with constrained arguments

In many cases, the arguments will be constrained. For instance, a set_display() function can be called for one specific device in a list, like ‘tv‘ and ‘projector’. In Python, we indicate this via string literals. In a CFG, we enforce this with pipes (“|”).

Here’s what such a grammar could look like for a function with type signature set_display(screen: Literal[‘tv’, ‘projector’]):

display_grammar = """
    ?start: "set_display(device = " device ")"
    ?device: "'tv'" | "'projector'"
"""

Note the double quotations wrapping ‘tv‘ and ‘projector’: the outer double quotations indicate that everything inside should be treated as a constant in the CFG. The inner single quotations indicate that it should be a string in the Python function call.

A similar effect can be achieved for Boolean arguments. Let’s add an argument “night_mode” to control how bright to set the display. For this new version of the function with signature set_display(screen: Literal[‘tv’, ‘projector’], night_mode: bool), we’ll generalize our notation a bit:

display_grammar = """
    ?start: "set_display(" arguments )"
    ?arguments: "device = " device ", night_mode = " night_mode
    ?device: "'tv'" | "'projector'"
    ?night_mode: "True" | "False"
"""

Example 3: One function with mixed constraints

Constrained generation also works with optional arguments. Suppose the smart home can close the blinds, either entirely or some fraction of the way. By default, calling the function close_blinds() closes the blinds completely, but the fraction can be controlled by an optional argument percentage which can take values 0 through 100. We can create a grammar for this function with signature close_blinds(percentage: int = 100) via:

percentages = list(range(101))
percentages_options = ' | '.join(percentages)

blinds_grammar = f"""
    ?start: "close_blinds(" arguments )"
    ?arguments: ("percentage = " percentage)
    ?percentage: {percentages_options}
"""

Example 4: Multiple functions

Thus far, we’ve treated each function and its grammar in isolation, but as we mentioned above, our smart home AI assistant would likely need access to all of these systems simultaneously. Each of the functions we’ve discussed could be treated as a tool. We can write a grammar that encapsulates a set of tools by adding some lightweight logic to connect the dots. If we wanted to allow just one function call at a time, we could pipe the function options:

percentages = [str(i) for i in range(101)]
percentages_options = ' | '.join(percentages)

one_function_grammar = f"""
?start: "<|tool_call_start|>[" function "]<|tool_call_end|>"

?function: save_note | set_display | close_blinds

?save_note: "save_note(" UNESCAPED_STRING ")"

?set_display: "set_display(" display_arguments ")"
?display_arguments: "device = " device ", night_mode = " night_mode
?device: "'tv'" | "'projector'"
?night_mode: "True" | "False"

?close_blinds: "close_blinds(" blinds_arguments ")"
?blinds_arguments: "percentage = " percentage
?percentage: {percentages_options}

%import common.UNESCAPED_STRING
"""

We can also accommodate multi-function-call responses in dotgrammar. While we’re at it, let’s wrap the (potentially multiple) function calls in begin <|tool_call_start|>and end <|tool_call_end|>tokens to simplify parsing:

percentages = [str(i) for i in range(101)]
percentages_options = ' | '.join(percentages)

multi_function_grammar = f"""
?start: "<|tool_call_start|>[\