/ LOCAL AUTONOMOUS AI
/ MACOS COMPUTER USE
/ NATIVE 3D SYNTHESIS
An emotionally expressive companion built to see, listen, reason, act, remember, and create beside you.
January links physical input, local state, a dynamic intelligence layer, and tangible output in one continuous system.
The system begins with the MacBook microphone, camera, terminal, and optional floor-plan imagery. Its daemon coordinates the resulting signal through listening, reasoning, memory, safety, and synthesis layers so a spoken thought can become an answer, a desktop action, a 3D scene, or a natural voice response without breaking the thread of the conversation.
Perceive → coordinate → reason → act
A 16 kHz microphone, native AVFoundation camera stream, terminal REPL, and blueprint files give January a direct view of both the room and the work.
Voice activity detection, Faster-Whisper, wake and sleep handling, and a working state machine coordinate passive, listening, working, speaking, and sleeping modes.
Dynamic routing, web and weather access, coding tools, emotional context, and model fallbacks route each task to the capability that best fits it.
January can speak through the Mac, take bounded computer actions, create Blender assets, or return detailed answers in the terminal.
January’s router is designed to preserve momentum when a task changes, a model is preferred, or a provider needs a fallback.
Every request passes through an intent check. A user can lock a chosen model for the session, ask what model is active, browse free choices, or reset to automatic routing. In automatic mode, January begins with its primary Gemini tier, falls through to a second Gemini tier when needed, then selects from an indexed OpenRouter and OmniRoute capability catalog spanning reasoning, vision, coding, fast response, and zero-cost models.
Lock a model, or let January select the fit
Gemini Flash and Pro provide the default path for responsive multimodal work and general intelligence.
A second Gemini key keeps the conversation moving through quota or provider interruptions.
The 458+ model catalog can select specialist pools such as Qwen and DeepSeek for code, Gemini and vision models for camera work, or R1-class reasoning models for planning.
Commands such as “switch model to DeepSeek R1,” “list free models,” and “reset model” make the routing decision visible and steerable.
January translates an instruction into visible macOS action while keeping a safety interlock at the center of the path.
The computer-using agent does not treat the desktop as an abstract API. It generates humanized quadratic Bézier cursor paths, physical clicks, drag strokes, natural per-character typing, shortcut chords, and managed application windows. Before any action, the SafetyInterlock checks native screen bounds and the emergency halt condition: moving the pointer to the top-left corner or pressing Ctrl+C stops the active sequence immediately.
Every desktop action passes through the same safety gate
The top-left corner at x ≤ 10 and y ≤ 10 is a global stop zone. Coordinate clamping prevents actions from leaving the physical display.
Mouse movement follows smooth Bézier trajectories with small deviations instead of teleporting between points.
Typing uses a varied cadence, while shortcuts can execute complete chords such as cmd+s or cmd+shift+p.
Window management can launch an app, bring it forward, center it, and keep the action legible on screen.
January turns a request into a procedural Blender scene, then hands the result back in formats made to travel.
A 3D request is interpreted as an object or scene intent, then converted into procedural mesh instructions, PBR material choices, studio lighting, a 35 mm presentation camera, and a Blender project. January can build cyberpunk swords, mugs, sports cars, chairs, vases, and torus sculptures, then save the native .blend project alongside universal .obj/.mtl and realtime .glb exports before opening Blender on the Mac.
Describe a thing → synthesize it → open it natively
Generate dual-tone plasma swords, solidified ceramic mugs, low-poly vehicles, minimal chairs, porcelain forms, and parametric torus knots.
Principled shading defines metallic, roughness, specular, glass, and emissive behavior rather than leaving models as plain geometry.
A three-point studio setup with key, rim, fill, and a composed camera makes every generated object immediately readable.
Each asset is written as .blend for Blender, .obj + .mtl for universal interchange, and .glb for web and realtime contexts.
For real machines and ambitious custom forms, January shifts from a preset to specification-led synthesis.
January’s technical grounding engine first looks for a curated iconic reference, then can extract dimensional telemetry from the web for unfamiliar subjects. Its BMesh kernel constructs station-lofted fuselages, NACA and supercritical airfoils, sweep, dihedral, washout, nacelles, windows, cockpit glazing, PBR surfaces, and studio presentation. The universal engine extends the same approach beyond aircraft to villas, appliances, instruments, tools, props, vehicles, and arbitrary multi-component objects.
Research → normalize → loft → present
Known subjects such as a Boeing 787-9, F-22 Raptor, Concorde, Spitfire, or Cessna can begin from curated engineering dimensions and materials.
Station skinning and mathematical airfoil curves create clean, dimension-aware surfaces instead of a generic approximation.
A modern villa, retro toaster, surveillance drone, electric guitar, or microscope is decomposed into a clear hierarchy of manufacturable forms.
January can inspect technical specifications and reference imagery before generating the Blender scene for a detailed request.
January reads spatial intent from a floor plan and constructs a usable architectural scene in Blender.
A camera snapshot, CAD export, or 2D floor plan enters the multimodal plan analyzer for structural topology and room decomposition. The architectural bridge then lays foundations, PBR floor zones, three-metre walls, openings, windows, doors, furniture, and lighting into a coherent scene. This is a direct route from a visual plan to an editable 3D building rather than a decorative image-to-image conversion.
See the plan → map the structure → build the space
Blueprint images are interpreted for rooms, walls, openings, orientation, and structural relationships.
The bridge creates foundations, wall runs, floor materials, windows, doors, and furniture from the extracted spatial layout.
Architectural elements receive physically based materials and daylight-ready scene settings for a convincing Blender presentation.
The finished scene is saved and launched in Blender, so the plan can continue into a real design workflow.
January can hold a lightweight local understanding of presence, context, and the rhythms that make an interaction feel continuous.
The native Swift camera engine captures a 1080p frame in under 0.6 seconds and can maintain a 60 FPS hardware stream. Local OpenCV face detection runs in under 20 milliseconds for presence and arrival awareness, while ambient monitoring can observe posture and gestures such as a wave. Alongside this perception loop, learned_profile.json and interactions.jsonl build an adaptive local record of preferences, patterns, and successful interactions.
Observe carefully, retain locally, respond with context
AVFoundation drives a hardware stream, while a Swift snapshot engine delivers fresh frames quickly for visual reasoning.
A lightweight OpenCV detector can identify presence and route face-aware behavior without sending the detection step away from the machine.
Arrival, posture, and wave gestures can become useful context for an interaction rather than a passive video feed.
January stores interaction history and learned preferences locally to improve continuity over time.
January’s SQLite brain gives long-running work and natural dialogue a durable local context window.
The persistent memory unit keeps identity, preferences, interaction moments, and active context in a structured SQLite database rather than a temporary chat buffer. A dedicated CLI exposes brain inspection and control commands, while REST endpoints make the stored context available to the local system. The purpose is simple: January should be able to pick up a working thread, recall a preference, or adapt its tone without making the user retell the essentials each time.
Remember the useful parts, locally and deliberately
Conversation continuity is anchored in a local relational memory unit with structured entries rather than only a transient model context.
Interests, working patterns, recurring details, and prior outcomes can inform future responses.
Brain-focused terminal commands make it possible to query, review, and manage stored conversational context.
Local REST endpoints connect the brain database to the rest of January’s daemon and interactive surfaces.
January can find, open, create, and explain work across the native desktop and a production-minded coding surface.
The macOS system-control engine uses Spotlight search, AppleScript, and normal application launching to bring requested files, folders, apps, media, and windows into view. Its code delegation path can generate Python, C, and C++ with type-aware, modern examples and practical run instructions. Live DuckDuckGo search and wttr.in weather access round out the system with current external context when a task needs it.
Ask for the thing, then keep working in the right place
Ask to launch VS Code or Safari, open a report, show Downloads, or play a project video; January searches and opens the appropriate macOS target.
The coding engine produces clean Python 3.10+, C99/C11, and modern C++20 responses, with language-appropriate structure and run commands.
DuckDuckGo-powered web search and weather lookups provide a current answer when local information alone is not enough.
Window management supports launching, focusing, centering, and preparing the application that the next task requires.
January tunes its spoken delivery to the moment while retaining a resilient path when a preferred voice service is unavailable.
A local emotion engine classifies tone and sentiment into seven mood archetypes, then provides that context to the response and voice layers. ElevenLabs delivers an ultra-realistic neural voice that can adjust prosody with the active emotional frame. If that path is unavailable, Microsoft Edge-TTS and the native macOS speech system preserve multilingual spoken output, while echo muting helps prevent January from listening to itself.
Understand the tone → choose the voice → speak naturally
The local classifier tracks sentiment and emotional context so a response can sound calm, alert, encouraging, reflective, or appropriately direct.
Neural voice delivery can be shaped in real time by the emotion engine instead of treating every answer as the same flat reading.
Microsoft Edge-TTS and native speech provide a multi-tier fallback path when the preferred neural voice is unavailable or offline.
Responses play through the MacBook speakers with echo-aware handling so room conversation can remain practical.
January recognizes language as part of an ongoing relationship, not a setting that needs to be chosen again for every message.
The language router identifies English and fifteen Indian languages from Unicode script and spoken context. A request such as “speak in Hindi” or “मराठीत बोला” can lock the session to the appropriate language, preserve the selected writing system, and choose the matching voice path. Phonetic transliteration handling helps bridge prompts that move between Latin characters and native scripts without breaking conversational intent.
Detect → retain → express in the chosen language
Unicode-aware classification identifies the language and script needed for a clear spoken or written reply.
Once a language is selected, January retains the preference across the conversation until the user asks to return to English or switch again.
Responses can use the appropriate Indian writing system rather than forcing a transliterated-only experience.
The selected script and language inform the voice router so spoken output remains aligned with the conversation.
January’s operating modes are designed around a room, a wake phrase, and the moments when a keyboard is simply the better interface.
In daemon mode, January runs headlessly with an energy threshold and a 1.2-second reverb guard, waiting for the wake phrase “Rise” and returning to sleep on “good night.” In terminal mode, the dual-section [ME] and [JANUARY] REPL temporarily pauses the background microphone while typed work is in progress, then restores room listening when the CLI exits. The same lifecycle controls camera eyes with explicit “eyes open” and “eyes closed” phrases.
Passive ⇄ listening ⇄ working ⇄ speaking ⇄ sleeping
npm run dev starts a headless listener that can hear wake phrases and spoken requests without a visible application window.
npm run cli opens a focused [ME] / [JANUARY] exchange and coordinates microphone ownership while the user types.
Voice activity detection and a reverb guard help distinguish a new spoken command from January’s own speech.
The project includes verification suites for CUA, BIM, precision and universal 3D, speech safety, voice integration, routing, and persistent memory.
The command surface stays conversational while exposing the full range of January’s model, camera, desktop, 3D, language, and system capabilities.
January’s command reference is deliberately direct. Ask “what model are you using?” to inspect routing, “make a 3D model of a cyber sword in Blender” to begin a scene, “look at what I’m holding” to invoke visual reasoning, “open VS Code” to control the Mac, or “what is the weather in Hubli?” for a live lookup. Commands can be spoken in the room or typed into the terminal, then are dispatched through the same state, safety, and memory layers shown above.
One command language across voice, terminal, and visual work
“Switch model to Liquid,” “switch model to Qwen Coder,” “list free models,” and “reset model” control the dynamic model-routing session.
“Eyes open,” “eyes closed,” “what do you see?” and “who am I?” control capture, visual query, and local face-aware behavior.
Request a sports car, coffee mug, Boeing 787-9, modern villa, floor plan conversion, toaster, drone, guitar, or any detailed object for Blender synthesis.
Move a cursor, type a phrase, open an application, locate a document, show a folder, play media, search the web, or retrieve live weather.
“Speak in Hindi,” “मराठीत बोला,” “switch to English,” “good night,” and “Rise” control language preference and the active listening lifecycle.
January is a macOS-native system with an explicit operating foundation: Node, TypeScript, Python, Blender, local data stores, and tested service boundaries.
The project is organized around a root package and a server workspace with dedicated audio, camera, vision, wake, memory, model, Blender, GUI, tool, and test modules. It requires macOS, Node.js 20+, Python 3.11+, Blender 4.0 or later, cliclick for computer-use actuation, and uv for Python package management. API keys and phrases are configured in server/.env, including primary and fallback Gemini access, ElevenLabs, OpenRouter, optional Claude, and voice/camera wake and sleep phrases.
Configure once, then choose room-scale or terminal-scale interaction
TypeScript coordinates the daemon, CLI, router, tools, and tests; Python powers audio and local emotion work; Swift provides native camera capture; Blender provides 3D synthesis.
Captures, face profiles, learned memory, model catalogs, and generated 3D exports live in dedicated server/data folders alongside focused engines for every system layer.
Install dependencies, compile with npm run build, then create server/.env from the example and configure model keys, voice settings, and wake/sleep phrases.
Dedicated suites cover computer use and Blender, plan-to-BIM, conversational 3D routing, engineering precision, universal synthesis, voice safety, fast fallback, and SQLite memory.
Use Ctrl+C, npx kill-port 3001, or the spoken and typed “good night” command to put January back into a quiet state.
The complete January map: how the system is arranged, how a request travels, and where each part lives.
Master system architecture
Microphone
Camera eyes
CLI + plans
VAD + Whisper
State machine
Local memory
Emotion engine
Model router
Web + code
CUA safety
Blender + BIM
Neural voice
Physical input and stored context converge in the daemon; January then routes the task to intelligence, tools, or synthesis before returning a spoken, on-screen, or generated result.
End-to-end sequence diagram
Voice activity detection receives a wake phrase or typed CLI input and moves January out of passive standby.
Faster-Whisper transcribes speech; camera or plan imagery enters the visual path when the request needs sight.
The state machine adds local memory and emotion context, then the router selects the model, tool, or actuation path.
January can answer, search, code, move the desktop, synthesize a Blender scene, or inspect the physical camera context.
The voice router produces the reply, while memory and interaction records retain the useful outcome for future continuity.
Project file stack
package.json · run scripts
README.md · system reference
openrouter_models.csv · model catalog
captures · latest camera frame
faces · enrolled profiles
memory · learned profile + interactions
models · catalog cache
exports/3d · Blender output
camera_engine · Swift + OpenCV
audio_engine · Whisper + TTS + emotion
src/models · dynamic router
src/cua + gui · desktop actions
src/blender + vision · 3D and perception
src/memory + emotions · continuity
src/audio + wake · listening lifecycle
src/gemini + tools · intelligence surface
src/tests · subsystem verification
The file stack follows the same boundaries as the architecture: data stays local, native engines are separated by modality, and the TypeScript source is divided into focused orchestration, intelligence, actuation, synthesis, and test layers.
JANUARY · local autonomous intelligence for macOS · Rise when you are ready
