You have to work on two concepts. People are stupid and won’t review the AI work and people are malicious.
It’s absolutely trivial to taint AI output with proper training. A Chinese model could easily just be trained to output malicious code in certain situation. Or be trained to output other specifically misleading data in critical situations.
Obviously any model has the same risks, but there’s an inherent trust toward models made by yourself or your geopolitical allies.
It’s impractical to approve and host every single model. Similar things happen with suppliers at big companies, they have a few approved suppliers as it’s time consuming to vet everyone
Might be nice is I could use that! We are stuck on default copilot with a crappy 64k context. It barfs all the time now because it updated itself with some sort of search function now that seems to search the codebase, which of course will full the context window pretty quick....
128k context has been a limiting factor in many applications. I frequently deal with data that goes upto 500-600k token range so i have to run multiple passes to first condense and then rerun on the combination of condensed. This makes my life easier.
Many SOTA models were already much more than 128k, namely 1M
Literally the only definitive SOTA model with 1M+ context is 2.5 pro. 2.0 thinking and 2.0 pro weren’t SOTA, and outside of that, the implication that there have been other major players in long context is mostly wrong. Claude’s had 200k for a second with significant performance drop off, and OpenAI’s were limited to 128k. So where is “many” coming from?
But yes, 10M is very good… if it works well. So far we only have needle in a haystack benchmarks which aren’t very useful for most real life performance.
basically it’s how many tokens (letters or group of letters) that the LLM can use as “context” in its response. 10M tokens is like, 7M words.
so, you could give Llama 4 a 7M word book and ask about it and it could summarize it, talk about it, etc. or you could have an extremely long conversation with it and it could remember things said at the beginning (as long as the entire chat is within the 10M token limit).
10M context is just absolutely massive - even the 2M context from Gemini 2.5 is crazy. Think huge code bases, an entire library of books, etc.
True but don’t tokens counts as characters and spaces not words? And the entire context window is a blend of input(your prompts) and output(ai response) tokens?
It's how many tokens (letters/words) the model can keep in its short term memory. When you go above that number in a conversation (or if you feed a pdf or code to a model that's too long), the model goes crazy.
(If I'm wrong on this, I'm sure reddit will let me know)
"Goes crazy" is a bit much, it just starts forgetting the earlier parts of the conversation.
The frustrating thing has always been that most online chatbot sites don't just tell you when it's happening, so you just have to guess and you might not realize the AI is forgetting old stuff until many messages later. Google's AI Studio site has a token count on the right and it's great, but having a colossal 10M context is also one way to get rid of the problem.
The context window is just the size of the input the model can accept. So if 1 word = 1 token (which is not true but gets the idea across), 10m context means the model could handle 10 million words of input at once. So if you wanted it to summarize many books, a few pdfs and have a long conversation about it, it could do that without missing any of that information in its input for each token it generates.
Why you should be hyped though? Idk be hyped about what you want to be hyped about. 10m context is good for some people, but not others. It depends on your use case.
When you start a chat with a model it knows a lot but doesn't remember anything you said in other chat. Context is "memory" it remember the thing you asked and the thing the ia answered. With this much contenx 6can upload a book or a paper and the model will know everything of it.
Important factor: context size is different from actual comprehension. It needs to both be technically capable of recalling info from 10M tokens ago and actually using them effectively (like Gemini 2.5 does, at least up to 120k)
It said Llama Scout above Gemma 3 and 2.0 flash lite, below 4o and 2.0 flash. So not really. Models that are o1 tier running locally are looking a couple months further out than I thought, hopefully by August. The mid tier and high tier models sound legit, but ain't no one running those on home systems.
I didn't say that, I meant these are not ready to use for coding on local personal computers yet, that's probably 4-6 months out for it to be o1 tier and actually usable.
4o is terrible at coding, and the current mid tier Llama 4 model has ~that accuracy, which requires a multi H100 card server to run. And Llama 4 scout (which is ~gemini 2.0 flash lite level, which is a joke capability wise) requires a single H100 to run the 4 bit quant.
We're still a ways off from high powered local models, but I think we should easily be there by September, latest by October.
I mean, we don't actually know how it performs at beyond 1 million tokens context. Like, theoretically, every model is infinite context if you don't account for performance past a certain point.
"Llama 4 Scout is both pre-trained and post-trained with a 256K context length, which empowers the base model with advanced length generalization capability."
I'm not getting excited until it's proven in a long context benchmark like the fiction.livebench. Older models had absolutely fake advertisements on this front.
No, but what it does mean is that we can expect all new foundation models from every lab to now be at or near that benchmark going forward.
Basically, this latest generation trained on a OOM more compute…Llama 4 is one of the first of that generation that is now coming to market at this new foundational context level, others will follow in tow.
Given Gemini's performance until 2.5 pro, almost certainly garbage above 100k tokens, and likely leaning into gibberish territory after 50k. Gemini's 1M context window was entirely on paper, this will likely play out the same, but hoo boy do I want to be wrong.
Yup that's what I do. I even have it analyze just one function and immediately roll to a new chat usually, the smaller the context the more accurate it is, so that's my go to strategy.
Wonder if this could beat Pokémon since it has such a huge context window - isn’t that the issue with Claude? Like, it couldn’t remember enough so it couldn’t get unstuck?
There haven’t been many benchmarks on actual recall and summary/application with this extended context length besides a needle in a haystack evaluation which can be a good preliminary (very basic) metric but not usually representative of many real world tasks. So we’ll have to see how well it holds up. Also, it’ll likely not be as smart as 3.5-3.7 Claude models. I’m excited to see how Gemini 2.5 pro does with this.
It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet.
Our goal is to remove bias from our AI models and to make sure that Llama can understand and articulate both sides of a contentious issue. As part of this work, we’re continuing to make Llama more responsive so that it answers questions, can respond to a variety of different viewpoints without passing judgment, and doesn't favor some views over others.
We have made improvements on these efforts with this release—Llama 4 performs significantly better than Llama 3 and is comparable to Grok:
Llama 4 refuses less on debated political and social topics overall (from 7% in Llama 3.3 to below 2%).
Llama 4 is dramatically more balanced with which prompts it refuses to respond to (the proportion of unequal response refusals is now less than 1% on a set of debated topical questions).
Our testing shows that Llama 4 responds with strong political lean at a rate comparable to Grok (and at half of the rate of Llama 3.3) on a contentious set of political or social topics. While we are making progress, we know we have more work to do and will continue to drive this rate further down.
We’re proud of this progress to date and remain committed to our goal of eliminating overall bias in our models.
In other words, introducing less bias in during the fine-tuning stage will give a more accurate representation of the model (not to mention a more accurate reflection of the human population).
The question is always: What do the builders consider to be true what do they consider to be biased?
Some will say that recognizing transgender people is biased and some will say it is true. Given Zuck's hard turn to the right, I'm concerned about what his definition of unbiased is.
In order to turn an LLM into a chat bot you have to do reinforcement learning. This means you give the AI a set of prompts and answers then you give it prompts and rate its answers.
A human does this work and the human has a perspective on what is true and false and in what is good or bad. If the AI says the earth is flat then they'll mark that down and if it gets after and yells at the user they'll mark that down. An "unbiased response" is merely one that agrees with your own biases. The people doing reinforcement learning dummy have access to universal truth, and neither does anything else in the universe. So both the users and the trainers are going off their own concept of truth.
So a "less biased" AI is one that is biased towards its user base. So the question is, who is this user base that the builder was imagining when determining whether specific training responses were biased or not.
Almost every model has some sort of corporoate neoliberal bias that has pervaded Western culture. I'm not a fan of corporatism nor neoliberalism; in fact, would probably rather prefer a Chinese model over that.
If you think Zuckerberg took a "hard turn to the right" then you're one of those fringe nutjobs who is part of the problem. People should be concerned about AI that is aligned to any such fringe ideology.
Yeah, moron, I must be an an anti-DEI gamer because I don’t believe Zuckerberg is a hard right winger. The level of sheer stupidity among Reddit leftists is truly astonishing.
How humourous that you assume I'm a leftist. The Reddit gaymers have truly shallow and tribalistic political views.
Edit- Oh wait I just had to browse your comment history :P. Don't get mad that people can call you out for being predictable npcs.
"A coup of what? He’s already the head of the executive branch, including the military. One could also say it’s unprecedented that the military push modern DEI initiatives (those started under Obama) and many of those fired were known for pushing it. You’re just going to be definitively exposed as a nutcase when there’s no “coup”
Only an extreme leftist nutjob would think “This person doesn’t believe Zuckerberg is a hard right winger, therefore they must be a gamer who thinks DEI has ruined everything!”
And, of course, in true nutjob fashion, you dig through months of my comments to try to find any instance where I mentioned DEI. And notice that I actually gave no evaluation of DEI! I didn’t say it was good or bad, I simply said it was recent and the motivation for Trump’s actions in a specific context… and I was right!
So, thanks for demonstrating that you’re another reddit nutjob who is bad at logic. For your own health, you probably shouldn’t spend so much time and effort investigating a random person just to try to draw more tenuous connections. Go outside, my friend.
Its just for the record for the comment string where the sentiment is clear to see. If I were to ask you about the context of the comment thread where the quote is from it would go like this.
Me-Hey do you think Trump attempted a coup on janurary 6th?
You-Define coupe. From what we know there is no definitive legal statement that defines a coupe...
Unsurprisingly, the nutjob who thinks anyone who believes Zuckerberg is not hard right wing plays games and hates DEI, and who dug through months of comments of a random person to find any mention of DEI, also believes “the sentiment is clear to see” even though no sentiment can be derived from the words themselves.
That is extremely wrong, you should read up on Digital Colonialism and the “WEIRD” (western, educated, industrialised, rich and democratic) bias most if not all LLMs show due to their data set being predominantly Americanised and anglophone content. Right now; LLMs don’t show an unbiased view of the human population and although they are multilingual they are monocultural
Sure I can give you something to read. At the end you have to put the work in if you want.
Just to add. I was just rejecting your use of “more accurate reflection of the human population”.
Considering that more than 50% of the training data is English content is already a dead giveaway why LLMs are biased towards the American (western) culture…
Yes, dumb ass, an LLM that is less biased towards the far left or right of the American political parties *is* a more accurate reflection of the human population. And if you knew anything about logic, instead of just how to do a quick google search for the link you share, you would know that isn't inconsistent with the idea that LLMs are biased toward American culture generally.
10 million Tokens? What a time to be alive.
And people dont see the exponential growth.
Im getting hyped every day about new benchmarks and increasing graphs
Companies are playing sleight of hand with what they even mean by "model" these days, but the TL;DR here is that the context length they're advertising is only possible because they're generating summaries of it and throwing most of the information away.
They may have trained it with longer sequences, but that doesn't mean that the AI will ever even see all the information in an especially large context you give it. They're doing gymnastics to trim it down, hoping you won't notice the degradation.
Everyone is getting excited over the context limit but we don’t know how good it actually works. There is usually massive degradation after like 32k context.
Looks like it's still behind the new deepseek v3.1 in coding. Which means deepseek r2 is going to be absolutely insane. That's the model I'm waiting for. Maybe this is foolish but if I was forced to bet I'd go all in on r2 overtaking gemini 2.5. Openai better pray full o3 and o4-mini are good but I'm sure they are sweating.
livecodebench of 49 is decent for non thinking model. also it becomes apparent they are spending very high amounts just for another iteration of a huge teacher model, like gpt-4.5. it seems to be worth it in their circles. maybe we underestimate good base models completely. alternative explanation: they all gamble the same game and we stagnate. maybe they just have this kind of money... while i still work my ass off to pay rent
The models are being trained at 6 months cycle. Every 1-3% increment will take exponentially more compute. Hence, the LLMs have stagnated. The O1 training time accuracy plot for reference.
Whichever model is being used on meta.ai definitely sucks at writing. Hopefully it's scout. Feels like 3.1 or 3.3. Noticing no difference. It says it's llama 4, hopefully hallucinating.
Context is also horrific. 20 prompts in and it completely forgot the start of the session, telling me it can't read context from a previous session, lmao. The web version is total garbage and nerfed.
What a let down. The smallest model is 100+b and it got destroyed by qwq:32b in live code bench. I just hope deepseek and qwq figure out how to multimodel soon
Wild to see how fast small models are catching up. Llama 4 Scout is just 17B active params, runs on a single GPU, and still crushes benchmarks. Model size used to mean everything—now it’s all about smarter architectures and better efficiency
266
u/ExoticCard 1d ago
10M context??? OOOHHHH FUCK
Now we're really cooking