Working as a developer, I saw this common pattern around how we evaluate good vs bad in terms of any given scenario. And jotting down, it all comes down to context.
Is the model good? Does the person know about this? Do you understand?
All these above questions revolve around how much context you have. And all these concepts around skills, RAG, and agent file base start from it.
So why is context kept at a limit? Why is human context treated as a state of the art? And why can't AI replace the human brain concepts?
You see, context comes with a cost. The higher the context, the more the model consumes the token. And there is a higher chance your GPU won't be able to handle much context or can hallucinate.
The paradigm of AI seems to be at a rapid phase, and hopefully one day we can replicate human brain behavior in no-cost context.