Article
Finally, a Replacement for BERT: Introducing ModernBERT
- +13
bwarner, NohTow, bclavie, orionweller, ohallstrom, staghado, alexisgallagher, rbiswasfc, fladhak, tomaarsen, ncoop57, griffin, jph00, johnowhitaker, iacolippo
• • 751
🔄 In a Training Loop
David Smooke
Smooke
AI & ML interests
data, software, news, currency, cryptocurrency, software development, llms, internet usage, software market shares, startup investment data, startup location data, hackernoon
Recent Activity
upvoted an article about 13 hours ago
Finally, a Replacement for BERT: Introducing ModernBERT posted an update about 1 month ago
What the Cloud Actually Drinks from "AI’s Water Problem Is Smaller Than You Think" on @hackernoon https://hackernoon.com/ais-water-problem-is-smaller-than-you-think
For years, the environmental conversation about computing was about carbon. Water was an afterthought, buried in sustainability reports under "operational metrics." That was a mistake, because the physics were never subtle.
Here's how a conventional data center stays alive. Heat from the chips is carried out of the server hall by a closed loop of coolant, which dumps it into a second, open loop — the cooling tower — where water is sprayed into moving air and deliberately allowed to evaporate. It's a spectacularly efficient way to move heat, the same trick your body uses when it sweats. It is also, by definition, consumptive: that water doesn't go back to the river. It goes into the sky. Depending on climate and tuning, this costs between 1 and 9 liters of water per kilowatt-hour of server energy. posted an update 2 months ago
Say it with me, AI is moving beyond the phone, AI is moving beyond the phone, AI is moving beyond the phone. The weight Apple is putting on this says it all. They're fighting for the next hardware market before it hits the market. Harder to say what it means for OpenAI's consumer trust.
Re https://huggingface.co/apple vs. https://huggingface.co/openai: https://hackernoon.com/apple-v-openai-trade-secrets-lawsuit-the-42-most-explosive-allegations-and-the-internets-reactions
Organizations
upvoted an article about 13 hours ago
posted an update about 1 month ago
Post
110
What the Cloud Actually Drinks from "AI’s Water Problem Is Smaller Than You Think" on @hackernoon https://hackernoon.com/ais-water-problem-is-smaller-than-you-think
For years, the environmental conversation about computing was about carbon. Water was an afterthought, buried in sustainability reports under "operational metrics." That was a mistake, because the physics were never subtle.
Here's how a conventional data center stays alive. Heat from the chips is carried out of the server hall by a closed loop of coolant, which dumps it into a second, open loop — the cooling tower — where water is sprayed into moving air and deliberately allowed to evaporate. It's a spectacularly efficient way to move heat, the same trick your body uses when it sweats. It is also, by definition, consumptive: that water doesn't go back to the river. It goes into the sky. Depending on climate and tuning, this costs between 1 and 9 liters of water per kilowatt-hour of server energy.
For years, the environmental conversation about computing was about carbon. Water was an afterthought, buried in sustainability reports under "operational metrics." That was a mistake, because the physics were never subtle.
Here's how a conventional data center stays alive. Heat from the chips is carried out of the server hall by a closed loop of coolant, which dumps it into a second, open loop — the cooling tower — where water is sprayed into moving air and deliberately allowed to evaporate. It's a spectacularly efficient way to move heat, the same trick your body uses when it sweats. It is also, by definition, consumptive: that water doesn't go back to the river. It goes into the sky. Depending on climate and tuning, this costs between 1 and 9 liters of water per kilowatt-hour of server energy.
posted an update 2 months ago
Post
137
Say it with me, AI is moving beyond the phone, AI is moving beyond the phone, AI is moving beyond the phone. The weight Apple is putting on this says it all. They're fighting for the next hardware market before it hits the market. Harder to say what it means for OpenAI's consumer trust.
Re
apple vs. 
openai : https://hackernoon.com/apple-v-openai-trade-secrets-lawsuit-the-42-most-explosive-allegations-and-the-internets-reactions
Re
posted an update 3 months ago
Post
116
For forty years, the secret language that lets software talk to hardware was owned by a handful of companies. Then a free alternative escaped the lab and turned chip design into a matter of statecraft.
There is a sentence that security researchers, hardware startups, and the governments of at least three continents have all, in their own way, come to believe: you can put a company on a blacklist, but you cannot put a language on one.
That single insight is quietly rewiring the most concentrated industry on Earth. For more than four decades, the foundational vocabulary of computing — the instruction set architecture, or ISA, that determines how a piece of software actually speaks to the silicon beneath it — has been guarded like a crown jewel. In PCs, laptops, and servers, the proprietary x86 architecture built by Intel and AMD has reigned almost unchallenged. In phones and embedded gadgets, the British-born ARM architecture has been nearly as dominant. If you wanted to design your own processor, you paid millions in licensing fees, signed thick stacks of non-disclosure agreements, and accepted black-box silicon you could neither fully audit for hidden flaws nor bend to an unusual purpose.
That world is ending. Driven by the spread of open standards, the maturing of free chip-design software, and a global anxiety about who controls the supply chain, open-source silicon has vaulted from an academic daydream into a commercial reality. Processors designed collaboratively, in the open, are now booting Linux, shipping inside consumer laptops, rendering 3D graphics, and locking down the firmware in enterprise data centers.
This is the story of how that happened — and of why a royalty-free dictionary for talking to a chip became one of the most contested objects in twenty-first-century geopolitics.
Read Full: "The Open Chip Revolution Has Reached the Real World" https://hackernoon.com/the-open-chip-revolution-has-reached-the-real-world
There is a sentence that security researchers, hardware startups, and the governments of at least three continents have all, in their own way, come to believe: you can put a company on a blacklist, but you cannot put a language on one.
That single insight is quietly rewiring the most concentrated industry on Earth. For more than four decades, the foundational vocabulary of computing — the instruction set architecture, or ISA, that determines how a piece of software actually speaks to the silicon beneath it — has been guarded like a crown jewel. In PCs, laptops, and servers, the proprietary x86 architecture built by Intel and AMD has reigned almost unchallenged. In phones and embedded gadgets, the British-born ARM architecture has been nearly as dominant. If you wanted to design your own processor, you paid millions in licensing fees, signed thick stacks of non-disclosure agreements, and accepted black-box silicon you could neither fully audit for hidden flaws nor bend to an unusual purpose.
That world is ending. Driven by the spread of open standards, the maturing of free chip-design software, and a global anxiety about who controls the supply chain, open-source silicon has vaulted from an academic daydream into a commercial reality. Processors designed collaboratively, in the open, are now booting Linux, shipping inside consumer laptops, rendering 3D graphics, and locking down the firmware in enterprise data centers.
This is the story of how that happened — and of why a royalty-free dictionary for talking to a chip became one of the most contested objects in twenty-first-century geopolitics.
Read Full: "The Open Chip Revolution Has Reached the Real World" https://hackernoon.com/the-open-chip-revolution-has-reached-the-real-world
posted an update 3 months ago
Post
112
Vibe Coding Ends at Localhost https://hackernoon.com/vibe-coding-ends-at-localhost
Your AI agent can build the app. It has no idea how to ship it – and here's the structural reason why
I'm not a developer. I've spent twelve years as a CMO in tech, walking into software companies to figure out positioning, growth, and what the product should actually be. I'm not an engineer, but I love technology and I've always tried to live at the front edge of it — early on every tool, first to try the thing everyone's talking about a year later. I understand software from three angles: design, business, and customer. The fourth angle, the code, was always someone else's job.
For years I wanted to automate my own work. Build the little internal tools I kept describing to engineers who were too busy to build them. AI fixed half of that problem overnight. I could describe what I wanted in plain English and Claude would hand me a working application — real logic, a real interface, the whole thing running on my laptop.
__________________________
Vibe Coding Starts at Localhost https://hackernoon.com/vibe-coding-starts-at-localhost
I am a developer, a start-up founder, and a writer. I have been around this space since 2000, since the good old days of the web. Having returned from a 1-year journey of vibe coding, I leave this memo to coders and builders.
I’ll never forget that day, when we iterated together, again and again. On the side of the vibe, it wasn’t about not having challenges. It is just that things went through, in a flow. That day, when the battle was won, I felt voilá.
But before I could return to base, something changed inside me. All of a sudden, I turned to my droid and declared, "Hey, I just did all the work here!"
If you have been piloting with them, you know what happened next, that warm "Congratulations!" followed by "What can I do for you next?" Aren’t you listening buddy? So I turned back to human peers, broadcasting that I did all the work....
Your AI agent can build the app. It has no idea how to ship it – and here's the structural reason why
I'm not a developer. I've spent twelve years as a CMO in tech, walking into software companies to figure out positioning, growth, and what the product should actually be. I'm not an engineer, but I love technology and I've always tried to live at the front edge of it — early on every tool, first to try the thing everyone's talking about a year later. I understand software from three angles: design, business, and customer. The fourth angle, the code, was always someone else's job.
For years I wanted to automate my own work. Build the little internal tools I kept describing to engineers who were too busy to build them. AI fixed half of that problem overnight. I could describe what I wanted in plain English and Claude would hand me a working application — real logic, a real interface, the whole thing running on my laptop.
__________________________
Vibe Coding Starts at Localhost https://hackernoon.com/vibe-coding-starts-at-localhost
I am a developer, a start-up founder, and a writer. I have been around this space since 2000, since the good old days of the web. Having returned from a 1-year journey of vibe coding, I leave this memo to coders and builders.
I’ll never forget that day, when we iterated together, again and again. On the side of the vibe, it wasn’t about not having challenges. It is just that things went through, in a flow. That day, when the battle was won, I felt voilá.
But before I could return to base, something changed inside me. All of a sudden, I turned to my droid and declared, "Hey, I just did all the work here!"
If you have been piloting with them, you know what happened next, that warm "Congratulations!" followed by "What can I do for you next?" Aren’t you listening buddy? So I turned back to human peers, broadcasting that I did all the work....
posted an update 4 months ago
Post
157
Before 
chatgpt launched, ~5% of new web articles were AI-generated. By November 2024 that crossed 50%. By April 2025, 74% of new web pages contained AI-generated content.
What AI industrializes isn't bad content — it's plausible mediocrity. Grammatically correct, structurally coherent, superficially persuasive, and almost indistinguishable from average.
Researchers have already warned that AI-generated survey papers are flooding
arxiv-community — what was once a labor-intensive exercise in critical synthesis has become a low-barrier, high-volume output burying original work.
I wrote about why internet communities struggle to publish quality over quantity and what that means for every platform that doesn't actively resist it: https://hackernoon.com/why-internet-communities-struggle-to-publish-quality-over-quantity
What AI industrializes isn't bad content — it's plausible mediocrity. Grammatically correct, structurally coherent, superficially persuasive, and almost indistinguishable from average.
Researchers have already warned that AI-generated survey papers are flooding
I wrote about why internet communities struggle to publish quality over quantity and what that means for every platform that doesn't actively resist it: https://hackernoon.com/why-internet-communities-struggle-to-publish-quality-over-quantity
posted an update 8 months ago
Post
1183
New HackerNoon Post: The Words of Interest Benchmark Test For Matching an LLM to Your Interests https://hackernoon.com/the-words-of-interest-benchmark-test-for-matching-an-llm-to-your-interests
By picking individual words instead phrases or paraphrases or passages, this test bypasses plot summaries (which are everywhere regurgitating themselves online) and focuses on the author's words. It reveals whether an AI has truly "absorbed" the specific texture of a book or is simply echoing the general internet consensus.
By picking individual words instead phrases or paraphrases or passages, this test bypasses plot summaries (which are everywhere regurgitating themselves online) and focuses on the author's words. It reveals whether an AI has truly "absorbed" the specific texture of a book or is simply echoing the general internet consensus.
posted an update 11 months ago
Post
190
🚀 Building or experimenting with AI? Find free compute & software credits from major & emerging programs→ https://hackernoon.ai/software-compute-grants-credits-search 💰 $50M+ in active programs for AI developers, projects, and startups.
published an article 12 months ago
Article
Why Does MCP Matter? What is Google's Strategic Approach to MCP?
Smooke
• posted an update about 1 year ago
Post
319
"How AI Reasoning Mirrors Borges' Library of Babel"
https://hackernoon.com/how-ai-reasoning-mirrors-borges-library-of-babel
Many people have the intuition that an LLM doesn't really understand what it's saying. It doesn't really reason, it has no intent to convey anything, and it lacks an innate distinction between truth and falsehood.
The intuition is sound, but not so easy to substantiate. If you have a bit of technical savvy, you can point out that an LLM is just a fancy autocomplete, or a "token predictor". This consolidates the intuition a little - predicting tokens (i.e. word fragments, short words, symbols…) does seem like a far cry from deliberation - but it remains an intuition, which can be reasonably questioned: could you finish this sentence if you didn't build up an understanding of what the paragraph is getting at?
Now, those with more charity towards our talking machines will argue that LLMs are still poorly understood "black boxes", and accuse the skeptics of reductionism: isn't Nature full of "mundane" mechanisms that give rise to emergent phenomena, with qualities that differ from their basic causes? Maybe the complex mathematical machinery behind token prediction somehow produces genuine understanding?
Eh, probably not.
The key thing to understand about the token predictor perspective is that it isn't actually reductionist - at least not in the way of calling the brain "just a clump of neurons".
It's a decent characterization of the high-level, functional definition of the model. Yes, the model itself is a huge ensemble of small parts, but they all work together to satisfy exactly one demand: given N previous tokens, predict the next one. The ability to complete this task is the emergent property you'd expect to be downstream from numerical data flowing through the model's layers.
But couldn't something extra still emerge out of repeated token prediction?
... https://hackernoon.com/how-ai-reasoning-mirrors-borges-library-of-babel
https://hackernoon.com/how-ai-reasoning-mirrors-borges-library-of-babel
Many people have the intuition that an LLM doesn't really understand what it's saying. It doesn't really reason, it has no intent to convey anything, and it lacks an innate distinction between truth and falsehood.
The intuition is sound, but not so easy to substantiate. If you have a bit of technical savvy, you can point out that an LLM is just a fancy autocomplete, or a "token predictor". This consolidates the intuition a little - predicting tokens (i.e. word fragments, short words, symbols…) does seem like a far cry from deliberation - but it remains an intuition, which can be reasonably questioned: could you finish this sentence if you didn't build up an understanding of what the paragraph is getting at?
Now, those with more charity towards our talking machines will argue that LLMs are still poorly understood "black boxes", and accuse the skeptics of reductionism: isn't Nature full of "mundane" mechanisms that give rise to emergent phenomena, with qualities that differ from their basic causes? Maybe the complex mathematical machinery behind token prediction somehow produces genuine understanding?
Eh, probably not.
The key thing to understand about the token predictor perspective is that it isn't actually reductionist - at least not in the way of calling the brain "just a clump of neurons".
It's a decent characterization of the high-level, functional definition of the model. Yes, the model itself is a huge ensemble of small parts, but they all work together to satisfy exactly one demand: given N previous tokens, predict the next one. The ability to complete this task is the emergent property you'd expect to be downstream from numerical data flowing through the model's layers.
But couldn't something extra still emerge out of repeated token prediction?
... https://hackernoon.com/how-ai-reasoning-mirrors-borges-library-of-babel
liked 2
models about 1 year ago
upvoted a collection about 1 year ago
liked a
dataset about 1 year ago
Request to post
➕ 2
17
#36 opened over 2 years ago
by 
Walmart-the-bag
liked a
model about 1 year ago
liked 2
models over 1 year ago
posted an update over 1 year ago
Post
279
The AI Search Marketshare Race: OpenAI Accounts for 66.5% of 2.5M+ Monthly Citations to HackerNoon: https://hackernoon.com/the-ai-search-marketshare-race-openai-accounts-for-665percent-of-25m-monthly-citations-to-hackernoon
OpenAI (66.5%) accounts for the majority of AI search calls to HackerNoon, based on my analysis of end-user–initiated requests from AI assistant and AI search to HackerNoon blogs from April 22 to May 22, 2025. AmazonBot (Anthropic) followed with 25.5%, and Perplexity trailed at 8%. The total volume of end user AI requests jumped to 2,563,800 in 30 days, which is 34% more requests for HackerNoon blogs than my April report on the AI search marketshare — underscoring a growing dependence on AI-driven discovery.
Continue Reading: https://hackernoon.com/the-ai-search-marketshare-race-openai-accounts-for-665percent-of-25m-monthly-citations-to-hackernoon
OpenAI (66.5%) accounts for the majority of AI search calls to HackerNoon, based on my analysis of end-user–initiated requests from AI assistant and AI search to HackerNoon blogs from April 22 to May 22, 2025. AmazonBot (Anthropic) followed with 25.5%, and Perplexity trailed at 8%. The total volume of end user AI requests jumped to 2,563,800 in 30 days, which is 34% more requests for HackerNoon blogs than my April report on the AI search marketshare — underscoring a growing dependence on AI-driven discovery.
Continue Reading: https://hackernoon.com/the-ai-search-marketshare-race-openai-accounts-for-665percent-of-25m-monthly-citations-to-hackernoon