Hi Antoine! Interesting point about backend variance. Do you think serving layer... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

rebekkamikkoa 4 months ago | parent | context | favorite | on: Show HN: Forge – Guardrails take an 8B model from ...

Hi Antoine!

Interesting point about backend variance. Do you think serving layer should become part of standard LLM eval reporting?

zambelli 4 months ago [–]

Hi! Yes, I definitely think so. I've seen variance across all model families I looked at. The magnitude changes, but the presence of variance is a constant.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact