The best way to learn to write is to write and have someone critique you. That someone can be an AI it doesn’t change anything about the process, as long as the initial input is your own best effort and the final result is your own edit based on the feedback you received.
What a strange take. He’s a loser because he has an ex?
That’s an excellent point! On that topic I recently listened to an interview of the founder of EleutherAI, who focuses on training small language models. She said they were able to train a 1B parameters reasoning model with 50K Wikipedia articles and carefully curated RL traces. The thing could run in your smartphone and is at parity with much larger models trained on trillions of tokens.
She also scoffed at Common Crawl and said it contained mostly cookies and porn. She had a kind of attitude like “no wonder the big labs need to slurp trillions of tokens when the tokens are such low quality”. Very interesting approach, if you understand french I can only recommend the interview.
It’s been a while since I used any MS product but I’ve got the same feeling with Google products. Weird bugs are starting to accumulate and at the same time they’re cramming every corner with buttons for their new AI integrations, with no explanation of how they’re supposed to work. It’s a mess, the stuff they add in doesn’t even respect the original app design so they’re really starting to look like they’re put together with toothpicks and duct tape.
+1 pour sshfs c’est vraiment facile à mettre en place et c’est assez performant
Euh pour le coup je suis assez d’accord avec toi je ne vois pas ce que Yunohost peut t’apporter si tu ne veux pas nécessairement héberger des services.
Ce que tu décris peut être fait très simplement avec un partage Samba. Je recommanderais d’utiliser une distrib immutable comme NixOs, ça va être un peu plus fastidieux à mettre en place au démarrage mais au moins tu as un système qui va coûter 0 en maintenance et qui est garanti d’être identique à chaque reboot. Mais sinon une Debian de base c’est très bien si tu es plus à l’aise avec ce genre de distri.
Au niveau hardware si tu ne fais que du partage de fichier, la config que tu décris sera largement suffisante. Ton bottleneck sera la rapidité des disques durs mais bon les vieux HDD c’est largement assez rapide pour streamer de la vidéo (si ce n’est pas du 8K avec 5.1) ou de la musique sur un réseau local.
The Revolutions podcast is a gold mine. Have you listened to the latest season? You’re in for a big surprise if you haven’t :)
That’s probably cause you do it without noticing
But then the reality starts striking exactly how you laid it out but by that time you’ve already submitted your two weeks notice.
OK that’s a fair observation. Honestly my naive guess would be that they simply do not optimize mainline gpt models for the kind of use case you generally have on Api (tool use, multi-step actions, etc…). They need it to be a perky every day assistant not necessarily a reliable worker. Already on gpt-4 i found it extremely mediocre compared to the Claude models of the same time.
I think that’s a more likely explanation than model collapse which is a really drastic phenomenon. A collapsed model will not just fail tasks at a higher rate, it will spit garbled text and go completely off the rails, which would be way more noticeable. It would also be weird that Claude models keep getting better and better while they’re probably fed roughly the same diet of synthetic data.
There is definitely something happening to that effect. A lot of people have been pushing for sovereignty for quite some time and they have wind in their sails currently.
Ah ! Bien vu j’ai effectivement mal interprété le message
La bulle IA n’a (normalement) pas d’impact sur ce genre de composants d’ancienne génération. Par contre à ta place je m’inquièterais plutôt de savoir si il est vraiment possible de changer ce CPU ? Il me semble que les modèles avec U dans le nom sont soudés directement sur la carte mère. J’espère me tromper mais c’est un point à vérifier.
The switch you mention (from 4th gen to 5th gen GPT) is when they introduced the model router, which created a lot of friction. Basically this will try to answer your question with as cheap a model as possible, so most of the time you won’t be using flagship 5.2 but a 5.2-mini or 5.2-tiny which are seriously dumber. This is done to save money of course, and the only way to guarantee pure 5.2 usage is to go through the API where you pay for every token.
There’s also a ton of affect and personal bias. Humans are notoriously bad at evaluating others intelligence, and this is especially true of chatbots which try to mimic specific personalities that may or may not mesh well with your own. For example, OpenAI’s signature “salesman & bootlicker” personality is grating to me and i consistently think it’s stupider than it is. I’ve even done a bit of double blind evaluation on various cognitive tasks to confirm my impression but the data really didn’t agree with me. It’s smart, roughly as smart as other models of its generation, but it’s just fucking insufferable. It’s like i see Sam Altman’s shit eating grin each time i read a word from ChatGPT, that’s why i stopped using it. That’s a property of me, the human, not GPT, the machine.
I’m sorry but no, models are definitely not collapsing. They still have a million issues and are subject to a variety of local optima, but they are not collapsing in any way. It is not known whether this can even happen in large models, and if it can it would require months of active effort to generate the toxic data and fine-tune models on that data. Nobody is gonna spend that kind of money to shoot themselves in the foot.
Yeah i remember that Ed article ! I don’t think the technical aspects are relevant to the newer generation of models, but yeah of course any attempt to compress inference costs can have side effects : either response quality will degrade for using dumber models, or you’ll have re-inference costs when the dumb model shits its pants. In fact the re-inference can become super costly as dumber models tend to get lost in reasoning loops more easily.
Yeah that’s also something that you have to train for, i’m not super aware of the technicals but model routing is definitely important to the AI companies. I suspect that’s part of why they can pretend that “inference is profitable” as they are already trying to squeeze it down as much as possible.
I mean what’s the point of morging if you’re gonna do it intermittoucly?
To clarify : model collapse is a hypothetical phenomenon that has only been observed in toy models under extreme circumstances. This is not related in any way to what is happening at OpenAI.
OpenAI made a bunch of choices in their product design which basically boil down to “what if we used a cheaper, dumber model to reply to you once in a while”.
I’m sorry but this is bullshit. For basic storage you absolutely don’t need a lot of RAM or SSDs, older gen hard drives are extremely easy to find and very cheap. People have been hypnotized into believing they absolutey definitely need the latest gen of hardware, without realizing it’s useless, and it’s the only kind of hardware that knows shortages and high prices.