My most recent llama cpp build is b9543 and today I notice that my local models don’t reason in the server web interface. Prior to that, I was using b8996 where they do reason. In the web interface, I see no reasoning being shown. However, models do reason in llama-cli.

I tried with --reasoning on, --reasoning-budget -1, --chat-template-kwargs '{"enable_thinking":true'. I didn’t use these flags before as reasoning was working fine in b8996.

  • Schilling2304@thelemmy.clubOP
    link
    fedilink
    English
    arrow-up
    1
    ·
    6 hours ago

    I confirm the same and it works now. I set it to maximum because fewer reasoning effort tokens cuts it directly.

    Thanks