~10,000 tokens/sec. Most teams over-spec for inference and under-spec for eval and fine-tuning. Where you put GPU budget matters more than how much GPU budget you have.
nicornle