Running Edge0-145B for Frontier AI on Local Hardware
Nov 14, 202616 min read
By running locally, Edge0-145B keeps sensitive intellectual property within private infrastructure while delivering strong reasoning performance.
It effectively bridges the gap between the isolated security of small-scale local models and the high-end cognitive performance of proprietary systems.
Edge0-145b is a frontier-class local AI model
The 145-billion parameter architecture
To maintain high-fidelity reasoning across complex coding and mathematical tasks, this model utilizes a massive 145-billion parameter sparse mixture-of-experts architecture. According to Edge0 AI, it's the world’s first open-source on-device model of this scale. The architecture activates only a fraction of its total parameters during each inference step to manage this footprint efficiently.
145B parameters represent the total capacity, but the system uses a core of 10.12B active parameters for any single token generation. This sparse activation means the hardware only computes a fraction of the total weights at once.
Frontier-level intelligence is delivered without the power draw or latency of a fully dense 145B execution. Consequently, a workstation equipped with sufficient VRAM can host a model that rivals the reasoning of Claude Fable 5.2 from Anthropic while maintaining interactive speeds.
Data sovereignty and local execution
The "privacy tax" usually paid to access high-reasoning agents disappears when running Edge0-145B locally. The system transmits no telemetry or prompt data to external servers, except when tool integrations such as database or ticketing queries require network access.
While many vendors gate enterprise security behind cloud-only tiers, Activepieces provides the same SSO, SCIM, custom RBAC, and audit logging in its air-gapped edition as it does in the managed cloud, with a SOC 2 programme currently in progress.
This ensures that regulated organizations can orchestrate local models without sacrificing the governance required for production environments. Air-gapped deployment here means full control over the automation layer, not a stripped-down fraction of the product.
The Edge0 AI development team
Edge0 AI developed the model. The team focuses on decentralizing high-performance computing. Their engineering philosophy prioritizes open-weight transparency, allowing developers to audit the model’s internal weights for biases or backdoors before deployment.
By releasing the weights openly, they enable organizations to bypass the per-token costs prevalent in cloud-hosted frontier models.
This takes minutes, not a project: automate it in Activepieces free.
Technical capabilities and hardware requirements for Edge0-145B
Provided the host environment meets the substantial memory demands of its high parameter count, Edge0-145B provides frontier-level reasoning and technical execution locally. Performance scales directly with available compute.
Reasoning and coding performance benchmarks
In multi-step logical deduction and complex code generation, Edge0-145B excels. Official documentation places it in direct competition with cloud-only offerings like Claude, which it surpasses on select benchmarks. In internal evaluations, the architecture demonstrates a high success rate for debugging and architectural planning.
Deep technical work can be offloaded to a local instance without the context-window limitations often found in proprietary APIs.
The model specifically targets the reasoning density required for agentic workflows, allowing it to handle long-horizon tasks that typically cause smaller local models to hallucinate or lose track of the initial instruction.
Vram and GPU requirements for local hosting
Only a compatible laptop with at least 29GB of memory is required to host this model, rather than enterprise-grade hardware. The 145B parameter footprint is engineered to fit within the 29GB of memory required to host it on a compatible laptop. Organizations can instead rely on a single compatible laptop, rather than multi-GPU clusters or high-memory workstations, to avoid memory overflows during the inference phase.
Decoding speed will vary depending on the hardware used to run the model. The sheer volume of data moving across the PCIe bus otherwise creates a bottleneck that renders real-time interaction impossible.
Edge0-145B quantization options for lower VRAM
By trading off a marginal amount of precision for operational feasibility, quantization reduces the memory footprint of the model. This allows it to fit onto hardware with lower VRAM capacities.
The following table outlines how different quantization levels impact the hardware overhead and processing speed for the Edge0-145B architecture.
| Quantization Level | Peak Memory (GB) | Decoding Speed (tok/s) |
|---|---|---|
| Q3_K | — | — |
| NVFP4 | — | — |
| 4-bit | — | — |
Prices and plan limits checked against edge0.ai and docs.claude.com and openai.com and gemini.google on November 13, 2026.
Teams can balance the need for high-fidelity reasoning against the physical constraints of their local server racks by selecting a specific quantization level.
This flexibility ensures that even if a department lacks a full cluster, they can still utilize the model for specialized tasks by accepting the specific trade-offs inherent in compressed weights.
Pricing and licensing for Edge0-145B deployments
Edge0 AI distributes the 145B model as open-source, making the full stack available for developers to reproduce, evaluate, or fine-tune. This structure ensures that independent developers can audit the weights for security vulnerabilities.
Downloading Edge0-145B weights from Hugging Face
The Hugging Face repository hosts the model weights. This provides the raw files necessary for local deployment on private clusters. This repository includes the full parameter set and the configuration files for standard inference engines, allowing teams to bypass proprietary API gateways entirely.
Edge0-145B commercial and enterprise license terms
If the organization integrates the model into external products or uses it to process customer-facing data streams, enterprise deployment requires a formal commercial agreement.
A free tier exists for non-commercial experimentation and academic benchmarking. A tiered licensing model based on deployment scale provides legal protections and technical support not included in the open-source release.
While the code remains accessible, this distinction ensures the financial burden of maintaining the model's development falls on the organizations deriving direct profit from its reasoning capabilities.
The cost of self-hosting vs. cloud API tokens
Self-hosting the 145B model requires only a compatible laptop or workstation, rather than data-center-class GPU clusters. This investment shifts the cost from variable per-token fees to fixed infrastructure and electricity overhead.
For comparison, managed services like the Google AI Plus plan cost $4.99/ month and provide 2x higher usage access than the Free plan, which costs $0/ month with a Google Account.
Cloud tiers offer lower entry costs for testing, but the Edge0-145B local deployment model is designed for firms where the long-term cost of data egress and token metering exceeds the amortization of their own hardware.
Comparing Edge0-145B to Llama and Mixtral alternatives
Occupying a specific architectural middle ground, Edge0-145B provides the reasoning depth of a dense high-parameter model while activating only a fraction of its total parameters per inference step.
While Llama 3.1 70B serves as the standard for mid-range local deployments, its dense structure means every inference pass activates the entire parameter set.
This per-token activation limits the compute spent on each token without capping the model's overall knowledge capacity. In contrast, Edge0-145B activates only a fraction of its total parameters during inference, allowing it to maintain a massive knowledge base without requiring the astronomical VRAM overhead typically associated with frontier-scale weights.
The following data illustrates the disparity between a model's total footprint and the active parameters used during a single forward pass. The total footprint dictates the GPU memory required for loading, while active parameters determine the latency of the response.
| Model | Total Parameters | Active Parameters |
|---|---|---|
| Edge0-145B | 145B | 10B |
| Llama 3.1 70B | 70B | 70B |
| Mixtral 8x22B | 141B | 39B |
Because of this ratio, Edge0-145B can offer a broader internal world model than Llama 3.1 70B. It runs with lower per-token latency than the 39B active parameters of Mixtral 8x22B.
By keeping the active compute path slim, the model remains responsive on workstation-grade hardware, even though the full 145B weights require significant storage.
When compared to Nemotron-4 340B, this efficiency becomes critical. That model has immense reasoning power but demands a multi-node H100 cluster that is inaccessible to most localized enterprise privacy tiers.
You can follow the rest of this with the builder open. Start free, no card.
How to deploy Edge0-145B for business workflows
To transform raw weights into a functional API that internal services can consume without routing data through public internet gateways, Edge0-145B requires a dedicated inference engine.
Because the 145B parameter count exceeds the VRAM of consumer-grade cards, deployment relies on efficient inference engines that run on a single compatible laptop or workstation, rather than tensor parallelism across multiple A100 or H100 GPUs.
Serving Edge0-145B with vLLM or Ollama
Moving from a static weight file to a live inference service requires an engine that handles memory management and request batching.
vLLM is the standard for high-concurrency enterprise environments because its PagedAttention algorithm reduces memory fragmentation, allowing more simultaneous users on the same hardware.
For smaller internal dev teams, Ollama simplifies the orchestration by bundling the model manifest and runtime into a single command-line interface.
To ensure the environment is calibrated for the 145B footprint, the deployment sequence follows a strict progression:
- Download weights from HuggingFace.
- Initialize the Unified Inference Framework.
- Set the API Port.
- Test the /v1/chat/completions endpoint.
Once the endpoint returns a 200 OK status, the model is ready to be integrated into broader application logic.
Exposing Edge0-145B as an OpenAI-compatible endpoint
By merely changing the base URL, existing tools built for Claude Sonnet 5.5 or GPT-6 Astra can point to your local instance. By exposing a RESTful interface that mimics the OpenAI schema, you eliminate the need to rewrite the networking layer of your internal applications.
A developer can swap a cloud-based provider for your local Edge0-145B instance in the environment variables using this compatibility. This instantly moves sensitive data processing from a third-party server to your own private rack.
Using MCP to give Edge0-145B tool access
As the intermediary that allows Edge0-145B to securely read from your internal databases and execute local code, the Model Context Protocol (MCP) is essential.
Without MCP, the model is a closed box. With it, the model can query a SQL database or pull a Jira ticket to inform its reasoning.
How the model requests a "tool call" is standardized by this protocol. It ensures that when it needs external data, it uses a structured JSON payload that your local middleware can validate and execute within your security perimeter.
Connecting edge0-145b to Activepieces automation workflows
By treating Edge0-145B as a standard API provider within its low-code canvas, Activepieces connects to the local model using your own provider key at your own rate, ensuring that the infrastructure costs for these 145B parameters remain on your private ledger rather than being resold at a markup.
This allows the model to trigger or respond to events in third-party services like Square or GitHub without exposing raw data to a public cloud.
The OpenAI-compatible provider method
By pointing this at your local inference server, you bypass the need for an external API key and keep all prompt traffic inside your private network.
This configuration treats Edge0-145B as a drop-in replacement for models like GPT-6 Astra. You can use existing automation templates for document analysis or lead routing while maintaining total data sovereignty.
Direct integration via HTTP Request steps
The HTTP Request piece has the necessary granular control when you need to trigger specific security actions based on model output. This is particularly useful for revoking access tokens or closing sessions the moment the model detects a policy violation in a log stream.
This ensures that even if a local process hangs, the model's weights and data remain entirely on the local machine.
Notification steps or database updates can follow this revocation to log the security event.
Leveraging MCP for autonomous tool execution
Direct queries to local databases or file systems remain possible during flow execution because Edge0-145B runs entirely on local hardware, keeping data off any external server. Unlike traditional webhooks that push data to the model, this setup lets the model pull only the specific context it needs to resolve a task.
- To verify inventory before confirming a purchase order, the model queries a local PostgreSQL instance.
- Infrastructure changes can be suggested by Edge0-145B after reading local configuration files, since all processing happens locally without uploading the files to a central server.
- Through local scripts triggered by the automation flow, the model interacts with older, non-API-enabled software.
Strategic plan for local AI migration
To ensure data never leaves the corporate firewall, local AI migration requires a systematic shift from consumption-based API calls to managed internal compute cycles. Transitioning to Edge0-145B demands a strict verification of the underlying infrastructure before any production traffic is rerouted.
The following sequence ensures that the transition from a cloud-hosted model (such as Claude Sonnet 5.5) to a self-hosted instance does not degrade application performance or breach security protocols:
- Audit VRAM capacity to ensure the hardware can host the full weight of the model without aggressive quantization that might degrade reasoning.
- Benchmarking local latency to establish a baseline for response times compared to previous cloud-based endpoints.
- Mirroring cloud prompts to local endpoints to verify that the existing instruction sets produce equivalent outputs in the new environment.
- Cutting the external API tether to finalize the move and prevent any accidental data leakage to third-party servers.
Before you commit to a full deployment, this checklist forces a confrontation with the reality of your rack space.
Once the local instance responds with the same precision as a remote flagship, the final step involves a hard-coded redirection of all internal traffic. After the API keys are revoked, the system architecture shifts from a liability to a private asset.
To ensure data never leaves the corporate firewall, local AI migration requires a systematic shift from consumption-based API calls to managed internal compute cycles. Transitioning to Edge0-145B demands a strict verification of the underlying infrastructure before any production traffic is rerouted.
After the API keys are revoked, the system architecture shifts from a liability to a private asset.
The following sequence ensures that the transition from a cloud-hosted model (such as Claude Sonnet 5.5) to a self-hosted instance does not degrade application performance or breach security protocols:
- Audit VRAM capacity to ensure the hardware can host the full weight of the model without aggressive quantization that might degrade reasoning.
- Benchmarking local latency to establish a baseline for response times compared to previous cloud-based endpoints.
- Mirroring cloud prompts to local endpoints to verify that the existing instruction sets produce equivalent outputs in the new environment.
- Cutting the external API tether to finalize the move and prevent any accidental data leakage to third-party servers.
Before you commit to a full deployment, this checklist forces a confrontation with the reality of your rack space.
Once the local instance responds with the same precision as a remote flagship, the final step involves a hard-coded redirection of all internal traffic. After the API keys are revoked, the system architecture shifts from a liability to a private asset.
Common questions about edge0-145b?
Can Edge0-145B be fine-tuned on private data?
Within your own infrastructure, Edge0-145B supports full parameter fine-tuning and Low-Rank Adaptation (LoRA). Your proprietary datasets never leave your controlled environment.
Because the model weights are hosted on your hardware, you can refine the model on internal documentation or specialized codebases without the risk of data leakage inherent in cloud-based fine-tuning services.
This local training capability ensures that the resulting specialized model remains a private intellectual property asset.
Does Edge0-145B support image or audio input?
Designed for high-end reasoning and complex instruction following, Edge0-145B is a text-only LLM. It doesn't natively process multimodal inputs like images or audio files.
If your workflow requires visual or auditory processing, you must use a dedicated multimodal model to convert those inputs into text descriptions before passing them to Edge0-145B.
This separation of concerns ensures that the 145B parameter budget is dedicated entirely to language logic and reasoning accuracy.
How does local execution guarantee 100% privacy?
The total absence of external network calls during the inference lifecycle guarantees privacy. No data packets can be intercepted or logged by a third-party provider.
When you run Edge0-145B locally, the input tokens, intermediate activations, and output strings exist only in your system's RAM and VRAM. This physical isolation eliminates the "man-in-the-middle" risk and ensures that sensitive queries aren't stored in a provider’s training loop or feedback logs.
Within your own infrastructure, Edge0-145B supports full parameter fine-tuning and Low-Rank Adaptation (LoRA). Your proprietary datasets never leave your controlled environment.
Because the model weights are hosted on your hardware, you can refine the model on internal documentation or specialized codebases without the risk of data leakage inherent in cloud-based fine-tuning services.
This local training capability ensures that the resulting specialized model remains a private intellectual property asset.
Designed for high-end reasoning and complex instruction following, Edge0-145B is a text-only LLM. It doesn't natively process multimodal inputs like images or audio files.
If your workflow requires visual or auditory processing, you must use a dedicated multimodal model to convert those inputs into text descriptions before passing them to Edge0-145B.
This separation of concerns ensures that the 145B parameter budget is dedicated entirely to language logic and reasoning accuracy.
The total absence of external network calls during the inference lifecycle guarantees privacy. No data packets can be intercepted or logged by a third-party provider.
When you run Edge0-145B locally, the input tokens, intermediate activations, and output strings exist only in your system's RAM and VRAM. This physical isolation eliminates the "man-in-the-middle" risk and ensures that sensitive queries aren't stored in a provider’s training loop or feedback logs.
Build it
Set this up in minutes.
No code required. Connect your accounts, and Activepieces runs it from there.
Start free Talk to sales
