AI Assistant JustAsk Your AI App Industries How it works Case studies Pricing Help Careers Sign in Build my AI assistant free
Dashboard AI Assistants Team assistant Knowledge library Team Profile Help
Theme
AI Engineering

AI Platform Engineer, Inference & Cost

Keep every answer fast and affordable: model routing, prompt caching, local inference for builds, usage metering and the fallbacks that keep chats alive.

Location
Surry Hills, NSW
Work
Hybrid or work from home (Australia)
Type
Full-time
Level
Senior

About the role

A Growth plan customer can send six thousand messages a month, and each one has to arrive in about a second for a fraction of a cent. You'll own the platform underneath: routing across hosted and local models, prompt-cache-friendly layouts, GPU inference servers for our build pipeline, rate limits, usage metering and graceful degradation when a provider has a bad day.

What you'll do

  • Own model routing, fallbacks and retries across providers
  • Run and tune local inference (llama.cpp-class servers) for site builds and demos
  • Track per-feature AI cost and latency, and drive both down release after release
  • Design credit and usage metering that is exact and never double-charges
  • Be the calm voice when an upstream API changes or goes down

What you'll bring

  • 5+ years in backend, infrastructure or platform engineering
  • Experience serving or integrating LLMs in production at meaningful volume
  • Deep understanding of latency, concurrency and caching
  • Comfort on Windows Server and Linux alike

Nice to have

  • GPU inference tuning (quantisation, batching, context slots)
  • Experience with OpenRouter, Vertex AI or similar gateways

How we work

You'll be part of a small team based in our Surry Hills office, L1 457 Elizabeth Street, Surry Hills, NSW 2010, Australia. Come in when it helps and work from home when it doesn't: the role is open to people anywhere in Australia. We agree on outcomes and a few shared hours, then trust you with the rest.

Apply for this role

AI Platform Engineer, Inference & Cost · Surry Hills, NSW · hybrid or work from home

CV (PDF or Word, up to 8 MB)
All open roles

More open roles

See all 14 roles
AI Engineering Hiring now

Senior AI Engineer, Retrieval & RAG

Own the retrieval layer every owns.ai answer stands on: hybrid search, chunking, ranking and the evaluations that prove each change made answers better.

  • Surry Hills, NSW
  • Hybrid · Work from home
  • Full-time
  • Senior
Posted 29 SepView role
AI Engineering

Graduate AI Engineer (2027 intake)

Start your career shipping real AI features to thousands of businesses, with a mentor, a structured first year and code in production from your first month.

  • Surry Hills, NSW
  • Hybrid · Work from home
  • Full-time
  • Graduate
Posted 7 OctView role
AI Engineering

Applied AI Engineer, Conversational Agents & Bookings

Teach assistants to get things done: book appointments, take call-backs and send jobs to Jobber, ServiceM8, Housecall Pro and simPRO without a human in the loop.

  • Surry Hills, NSW
  • Hybrid · Work from home
  • Full-time
  • Mid-level
Posted 6 OctView role