I wonder what is the game plan for OpenAI and Anthropic now. Suddenly their offering looks very bleak, Llama offering almost the same results for a fraction of the cost (for a large subset of tasks in English). Additionally it can be ran on-prem which is a very big plus for many companies that don't like their data leaving premise. To add to that it's constantly getting better being iterated over by the community.
And the 400 billion one is still in the making, extrapolating from what we've seen from the 8 and 70 billion ones it should beat the current commercial SOTA. Whatever OpenAI releases this year will need to be next-level for them to stay in the game as the leader.
No idea. The markets did lunch Meta in the gut today though so who knows what the outcomes will be. It feels like a lot of goodwill but investors don’t agree (even though EPS was up)
I've been using Llama 3 8B locally with Ollama since it was launched mostly for coding and other technical work. It's really great - I don't miss ChatGPT at all. Admittedly I didn't have the pro subscription, but Llama 3 is as good as the free version if not better.
I wonder what is the game plan for OpenAI and Anthropic now. Suddenly their offering looks very bleak, Llama offering almost the same results for a fraction of the cost (for a large subset of tasks in English). Additionally it can be ran on-prem which is a very big plus for many companies that don't like their data leaving premise. To add to that it's constantly getting better being iterated over by the community.
And the 400 billion one is still in the making, extrapolating from what we've seen from the 8 and 70 billion ones it should beat the current commercial SOTA. Whatever OpenAI releases this year will need to be next-level for them to stay in the game as the leader.
No idea. The markets did lunch Meta in the gut today though so who knows what the outcomes will be. It feels like a lot of goodwill but investors don’t agree (even though EPS was up)
The nice thing is that you can now use Llama 3 70B for very cheap ($0.9/M tokens) using FireworksAI, TogetherAI or similar providers.
have you tried https://groq.com/? I get ~800 tokens per second.
With Llama 7 8B, Groq still has high API limits, so you cannot use it at scale for a project.
I've been using Llama 3 8B locally with Ollama since it was launched mostly for coding and other technical work. It's really great - I don't miss ChatGPT at all. Admittedly I didn't have the pro subscription, but Llama 3 is as good as the free version if not better.
I wish I was having such results. GPT seems like the only option up to date / looks up docs.
> 1-2y old really kills you on modern front end development tooling
do you use the command line with ollama? I find formatting prompts a pain there.
I usually use the LLM CLI(1) or the Raycast Ollama extension (2). The first one works well enough for when I have to paste multiple lines.
1. https://github.com/simonw/llm?tab=readme-ov-file
2. https://www.raycast.com/massimiliano_pasquini/raycast-ollama
"Llama 3 saved my startup"
https://twitter.com/levelsio/status/1782875054864282104