Connecting Together AI
Together AI's serverless inference API is OpenAI-compatible, so you point an OpenAI client at gateway.palmsouqltd.com/together/v1, keep your Together API key, and add the PALM SOUQ headers. Model identifiers such as meta-llama/Llama-3.3-70B-Instruct-Turbo pass through unchanged.
Together hosts a large catalog of open-weight models with different prices. PALM SOUQ resolves the model id on every request against its price table, so switching a feature from one Llama size to another changes the cost per call in reports immediately.
Dedicated endpoints and allocation
If you run a dedicated endpoint on Together, you pay for the hardware by the hour rather than by the token. As with Bedrock provisioned throughput, PALM SOUQ can allocate that hourly cost across tenants by their share of tokens, so customer-level margin stays meaningful on reserved capacity.
Serverless and dedicated traffic can sit behind one PALM SOUQ route. A routing rule can send Enterprise tier tenants to the dedicated endpoint and everyone else to serverless.
Open-weight models as cost floors
The value of an open-weight host in a governance setup is that it gives you a cheaper floor to fall back to. When a Free tier tenant exhausts its monthly allowance, the gateway can serve them from a smaller Llama model on Together instead of returning an error, which keeps the product usable while the cost of that tenant stays bounded.
Every fallback decision is written to the audit log with the original model, the serving model and the budget state that caused the switch, so product and finance can see how often degradation happens and for whom.
Rolling it out safely
Run the price table check in the console for each Together model you use. It compares the gateway's rate for that model id with the current Together list price and flags any mismatch. With a large open-weight catalog, this catches a model swap that silently changed your cost per call.
If you run dedicated endpoints, upload the hourly rate so allocation by token share starts from day one, and compare the allocated total with your Together invoice at the end of the first month.
Fine-tuned models hosted on Together can be priced in PALM SOUQ with their own per-token rate. That way a tenant served by a custom fine-tune shows its real cost, and fine-tune spend can be compared with the base model it replaced on the same feature.
