r/quant • • 5h ago

Models How do you tell a well-calibrated forecaster from a lucky one after only ~50 calls?

0 Upvotes

been comparing a few public btc range forecasts, the kind that say "80% chance it closes between X and Y in 24h". after about 50 days most of them look fine, hit rates between 72% and 86%.

problem is 50 calls is nothing. a forecaster thats actually calibrated and one thats just lucky look the same at this sample size, and the band around 80% on 50 tries is huge.

things i've been considering:

  • pool across a few confidence levels (50/80/90) to get more signal per day
  • score the width too, so a lazy wide range doesnt win
  • interval score instead of plain hit rate

what do you actually use when the sample is this small? or do you just wait 6 months


r/quant • • 9h ago

Job Listing Experienced quant in no man’s land.

21 Upvotes

Dear all,

Quant lead at a small firm. Not even getting interviews at a scale I thought I would. None of the big firms even give me a OA

Every interview I’ve had has gone incredibly well but im not even hearing back about next steps.
I’ve even had them literally say we will get you to the next round and I just have silence.

Quite upsetting.