Activity Feed

AI & ML interests

None defined yet.

Recent Activity

appvoid 
posted an update about 5 hours ago
view post
Post
27
Byte-level state-space models. That sounded pretty scary for a scientist decades ago. Now we have:

1. Knowledge that deeper layers train smoothly.
2. Knowledge that Transformers work but is quadratic on sequence length.
3. Knowledge that SSMs work even better. Numerically unstable sometimes.
4. Speculative-decoding.
5. Open high-quality data.
6. Knowledge that KD works.

It slowly feels like is no longer a bad idea.
  • 1 reply
·
appvoid 
posted an update 2 days ago
view post
Post
116
It's 2026 and there are no instantaneous/fast vision language models for cpus yet. That's another free idea.
  • 6 replies
·
appvoid 
posted an update 4 days ago
view post
Post
123
CaraArchive highlights a broader reality of putting data online: once something is publicly accessible, it becomes extremely difficult to guarantee that it will remain under your control.

If there is information or artwork that you absolutely do not want copied, archived, scraped, downloaded, or used by others, the safest option is still not to publish it publicly in the first place. That may sound obvious, but the internet was fundamentally designed to move and reproduce information, and there are countless ways to retrieve publicly accessible images:from ordinary browser tools and web scraping to automated or agentic systems.

That does not mean artists should simply accept every possible use of their work. Artists deserve meaningful control, attribution, compensation, and reasonable ways to express how their work may be used. But treating the technology and peopple using it itself as the enemy is unlikely to solve the underlying problem.

There probably isn't a technical solution that can make a publicly visible image simultaneously viewable by everyone and impossible to copy. The realistic goal should therefore be to create better norms, incentives, licensing systems, and tools around how that content is used.

Like, we can imagine a future where every artist gets his/her own credentials and some kind of fingerprint done just like blockchain works. But that requires substantial cooperation among organizations, companies and individuals.

Technology and art are not inherently opposing sides though.
appvoid 
posted an update 9 days ago
view post
Post
162
If you lack ideas for a cool model, here's one.

Train a model from scratch on wikipedia with one twist: the tokenizer changes the actual token ids used on every sample fed. If somehow still learns English, you have made an astonishing discovery.

You would have answered the question: Can a model learn human languages from structure alone?
  • 2 replies
·
appvoid 
posted an update 11 days ago
view post
Post
1956
Random corporate secret of tonight:

Try overfitting a tiny model on billions of high-quality datapoints: you can't. You can do 100 epochs and see the model still improving.

You're welcome.
  • 1 reply
·
appvoid 
posted an update 12 days ago
view post
Post
856
- GLM 5.2
- Flux 3
- New Qwen model
- New small model leaderboards
- Lots of people finetuning smol models.
- Some even under 12 year olds clauders are here (was not on my bingo card this year)
- ChatGPT's Sol became a lot faster this week
- LFM2.5 2.6b
- Kimi K3 (though only a few will run it)
- New Ling 3.0 Tiny
- New video model that is making south park videos?
- Deepseek v4 flash being more honest than bigger models
- The new model from meta


Everything Everywhere All At Once
  • 1 reply
·
appvoid 
posted an update 14 days ago
view post
Post
2381
I don't know if it was us or one of you guys or maybe all of us at once but lately we have seen a finetuning/pretraining explosion of models below 200m params and we can't be more happy about it keep coming tinkerers all of this is possible because of you!
  • 8 replies
·
appvoid 
posted an update 15 days ago
view post
Post
2899
If you want small models to be great again you should give a follow to people like @Banaxi-Tech or @Datdanboi25

These guys are rocking it with small models lately.

(They are not paying me to say that)
  • 22 replies
·
appvoid 
posted an update 17 days ago
view post
Post
181
byte-level
deep layers
diverse data
compact size
overfitting
grokking

I know the next best small model is somewhere in the intersection of these features.
  • 1 reply
·
appvoid 
posted an update 18 days ago
view post
Post
896
Do you prefer AI companies/individuals to release half-baked models weekly or an overpowered model a month?
  • 2 replies
·
appvoid 
posted an update 19 days ago
appvoid 
posted an update 24 days ago
view post
Post
617
i love reinforcement learning
  • 1 reply
·
appvoid 
posted an update 26 days ago
view post
Post
1412
A Small Model is All You Need. Meet palmer-006 (90M)

After 3 years of experiments, we are finally releasing our flagship tiny model: **palmer-006**.

If you are building for edge hardware, SBCs (Raspberry Pi, etc.), or low-power devices, this is for you. Inspired by Andrej Karpathy's idea of a self-contained "cognitive core," we wanted to see how much power we could pack into a sub-100M parameter footprint.

🧠 **How we "Palmerized" it:**
We believe in starting our experiments with the absolute strongest baseline possible.
1. Light fine-tuning on highly curated data
2. Model merging
3. Another light fine-tuning round
4. Adjusted Mamba for maximum token speed ⚡️

⚠️ *Note: This is a foundational language model. It has not been instruction-tuned yet!*

Also, since this needs instruction tuning next to become a chat assistant—**what dataset would you recommend we use for the instruct tune?**

---
🔗 **Quick Links & Info:**

* **License:** Open for research, education, hobby, and modification! (For commercial use/hosted APIs, shoot an email to nosoyhackercodigo@gmail.com. *PS: Donators can claim a free commercial license!*)

* **Attribution:** Built using AI tech from the Technology Innovation Institute (TII).

Can't wait to see what you build at the edge. Let me know your prompt completions below! 👇

appvoid/palmer-006
  • 10 replies
·
appvoid 
posted an update 27 days ago
view post
Post
204
...for tomorrow.
appvoid 
posted an update about 1 month ago
view post
Post
3338
Two big projects are open sourced soon. Get ready...
  • 11 replies
·