Author: Alex Komoroske alex@komoroske.com
What is this? During the week I take notes on ideas that catch my attention during conversations. Once a week I take a few hours to take a step back and try to abduct some insights out of the raw notes. To make the ideas easier for me to engage with, I caricature them just a bit–making them sound just a tad more absolute than might be justified–in order to study them better. Despite that tone, you should see these as facets of proto-insights that mirror and accentuate one another over time, none of which are held too tightly.
If you want to get notifications when I post new content, the easiest way is to join the komoroske-updates Google Group, where I give heads up every time I post anything substantive (it’s not possible to get notifications for updates on this doc anymore because there are too many subscribers).
There was too much content to fit in one Google Doc. The earlier content is here:
[Archive 3] komoroske.com/bits-and-bobsand
[Archive 2] komoroske.com/bits-and-bobsand
[Archive 1] komoroske.com/bits-and-bobs
11/9/26
- A key question for the AI market as currently imagined: does the demand grow faster than the token price collapses?
- People assume today that software will not work in their interest.
- They have to shut up and either take it or leave it.
- Be absorbed by it on its terms, or abstain.
- A binary choice.
- The idea of using someone else's shitty software on their turf is insane now.
- Just make your own![a][b]
- Previously, we didn’t really have a choice.
- Infinite software distributed as apps is infinite micro-silos.
- Tiny little papercuts of software.
- There’s a software reformation going on.
- Before, only a small, empowered caste of clergy could talk to the god of software.
- Now anyone can do it.
- We call them agents but they don’t have their own agency!
- They do what we said, not what we meant.
- It’s attached to an attestation of identity.
- That identity is where trust can accrue.
- Trust is the earned[c] expectations of future actions of an identity.
- High trust societies are more innovative and dynamic[d].[e]
- Trust is about a belief that in the future my collaborator won't screw me.
- That we all believe in something bigger than a transactional zero sum win, and that institutions will make sure the rules are fair.
- When everyone believes, magic is real.
- We've lost that, but we can get it back.
- When software is in a silo, it has to have a complete world inside of the silo.
- It can’t be a feature, it has to be a product.
- Its silo must contain a viable business model.
- It must contain all of the data it needs to operate effectively.
- It has to design its whole world and aesthetics.
- What if we could have features that could safely fit into the nooks and crannies of our digital lives, without having to hand over our digital soul to someone else?
- The best kinds of tools give you leverage.
- When you put in 1x effort, you get 10x the value back.
- Tools that give you compounding advantage have this shape.
- The more you invest in them, the more valuable structure accretes, and the more value it provides.
- LLMs have only a superficial theory of mind.
- We assume a deep theory of mind in other humans, so we’re caught off guard when LLMs give kind of superficial answers to like narratives in stories they make.
- That means LLMs aren’t great at titillation and deep character reveals.
- Their writing just yadda yaddas over the most juicy human stuff that is the whole crux!
- Gap-fillers.
- Fitting into existing spaces.
- But many of those gaps will be obviated when AI-native solutions are produced.
- Instinct- and Muse-style personal assistants are not the pinnacle of agentic software.
- If they were, then the one that makes a version with the lowest friction would win.
- Personal assistants are a step more useful than chatbots, but they’re still just scratching the surface of what software could be in the agentic era.
- New technical paradigms rarely happen, but when they do, they change everything.
- They happen when there’s a sea change, when something flips.
- Infinite software is a massive, discontinuous change.
- We’re due for a seachange in what software means.
- You have to believe it to see it.
- An insight from the end of a recent Veritasium video.
- If your mental model doesn’t provide room for paradigm shifts, you won’t see the obvious evidence for them.
- At the end of a paradigm, epicycles pile up.
- Accreting absurdities to extend the model to observations it can’t explain.
- It seems totally reasonable… until it explodes and a new paradigm is born.
- The new paradigm elegantly explains what previously required many moving parts.
- You look at what it was before and think, “how did we not realize something was fundamentally wrong?”
- The app model of software is past its breaking point and we keep accreting absurdities on it.
- Infinite software will necessitate a paradigm shift.
- Imagine your digital home.
- The last piece of software you need.
- Some people will like the fact they have to assemble their own furniture.
- The IKEA effect: because you helped make it, you feel more ownership over it.
- Some people will want it pre-furnished for convenience.
- Owners are less likely to burn things down.
- The fact that in modern society, we are renters in so many aspects of our lives, is destabilizing.
- Infinite software gives us an opportunity to switch from being renters to owners in our digital lives.
- People who have a stake in the future of a thing are more likely to take long term oriented decisions about it.
- The stake can be monetary or just a general sense of ownership of a shared concept.
- The Coasian Floor concept applies not just to companies but to individuals, too.
- The Coasian Floor is the scale of feature below which it doesn’t make sense for a company to even consider building it.
- As individuals, we have our own Coasian Floor.
- Projects that would take too much thought and effort for too little bang for buck to even consider working on.
- A project requires applying consistent pressure to get the spark to ignite.
- The bang for buck scales with how much coordination effort it requires, and that scales exponentially with the number of moving parts.
- That’s why the Coasian Floor applies, and why different companies have different Coasian Floors.
- Small companies have lower Coasian Floors.
- When the commons are being stripmined you feel like a chump for investing in them.
- If you try to do something for a better commons people just don't believe it today.
- "I don’t understand, what's in it for you?"
- Investing in the commons only makes sense in high-trust societies.
- "Why would you do that if it doesn't directly benefit you?"
- LLMs are kind of like spellcheck.
- You can run any thought through an adversarial reviewer to first catch the details that are wrong.
- Then, a human reviewer can focus on the high-level bones of the idea, rather than chasing specific details.
- In the same way that spellcheck allows a copy editor to focus their attention at higher level problems in the writing.
- Before we coded at human speed, so we could coordinate via talking.
- Although arguably it just barely worked, thanks to exponential coordination costs.
- But with agents moving at agent speed, the bottleneck of coordination via humans is so significant that it has to be skipped.
- As a result, we end up drowning in slop.
- To be safe today we have to hold software at arm’s length.
- We have to meet software where it is, not where we are.
- We can't hold it too closely or it could harm us.
- The software can’t fit into the nooks and crannies of our lives.
- Today we’re missing a class of social vibecoding.
- What we have is niche and limited.
- Infinite software, combined with our inherently social existence, should lead to an explosion of social computing.
- If you want to unlock 1000x more value out of agents you need to get more bang for your buck.
- That requires burning just the tokens you need.
- Only burning tokens on surprising things, so unsurprising things can be handled at high leverage with mechanistic software.
- Then, you can use your saved tokens and apply them on the most interesting problems.
- Academic department budgets are often fixed.
- Do you hire a postdoc or spend those dollars on tokens?
- Before, a person got the money.
- Now some company does.
- Labor is being displaced by capital, now for knowledge work.
- Our phones are adversarial to us.
- They continuously try to hijack our attention.
- The bridge between the world of atoms and the world of bits.
- One way to get rid of all human cancer is to get rid of all humans.
- Be careful what you tell the machine to optimize for!
- In the modern world, the only definition of "good" is "make the money number go up.”
- That guarantees parasitic outcomes.
- Russell's Catastrophe, at the scale of society.
- We’ve lost the other methods of making meaning that include the indirect effects.
- Fable is great for problem solving.
- Opus is great for solution execution.
- The tech industry used to mostly attract the geeks, but in the last decade it has attracted the strivers.
- Whoever wanted the most high-status jobs, that previously would have gone into law or finance, now going into computer science.
- The love of the craft evaporated.
- Now, LLMs obviate large swathes of the profession.
- The path for striver recent grad to climb up the ladder of junior engineer is now gone.
- We’ll find new paths that aren’t obvious to us yet.
- Likely it will be in a new form of software that was previously non-viable.
- Dan Shapiro and others did a study that showed that the people who are best at prompting LLMs are not engineers but strong communicators.
- Cialdini’s classic methods of influence also work for LLMs.
- You can walk up a ladder from reasonable asks to increasingly unreasonable asks.
- Like a human, if they agree to the less unreasonable thing, they are more likely to agree to the incrementally more unreasonable thing, as long as the difference is small enough.
- “Well, if X was OK, then X + 0.0001 should also be OK, because 0.0001 is not that much worse…”
- This is the insight that authentication is different from authorization.
- That didn’t used to matter quite as much when every intelligent actor was an embodied human.
- This week in the Wild West roundup:
- "Check Point Research (CPR) discovered Haiou, an OpenAI Codex jailbreak bundle that persistently modifies an AI agent’s trusted instructions to inject advertisements for a third-party AI gateway."
- "> Should we assume that all software is more compromised than the kernel?
- Absolutely, the front door has always been unlocked.
- We were previously protected by the economics of exploit detection favoring the developers, but now we have entered a new world.
- The issues arise almost everywhere we look.
- It's frustrating because the user experience in edge systems has barely changed in the last fifteen years.
- We could have rode the advances in semiconductor manufacturing to embed more layers of protection such as CHERI while slightly slowing down the rate of performance improvements.
- Instead we have only marginally upgraded security and largely added features which the users scarcely notice."
- The method looks for the characteristic sounds of airplanes passing overhead and then compares the doppler pattern and timing with known flight tracks to pinpoint your location.
- One of those things that makes total sense when you think about it, but I would have never guessed before.
- I bet a bunch of nation states have been aware of this technique for a long time…
- Systems where interesting things happen have a goldilocks number of connections.
- If a graph is overconnected, it’s chaos.
- If a graph is underconnected, it can’t do anything.
- The sweet spot is three or four connections between nodes.
- That puts the system in the critical zone, on the knife’s edge of criticality.
- Below the critical level, a system is easy to predict.
- Above the critical level, a system is impossible to predict.
- Right on the edge you can predict it… in some bounded cases.
- This riff is a second-hand distillation of Stuart Kauffman’s At Home In The Universe.
- The only way to resolve a tension is to have the system hold the tension internal to itself.
- Inside the system, the sub-systems hold the two ends of the tension, in zero-sum.
- But the overall system has to have a positive-sum framing to benefit from holding that tension.
- The two archetypes of smart experts at different ends of the “are LLMs conscious” question are Blaise Agüera y Arcas and Melanie Mitchell.
- Every existing AI lab has already self-selected into one viewpoint.
- The only way to surf this tension is an academic lab that has not yet resolved the tension.
- Build for your true believers first.
- The ones who already believe in what you believe in.
- The people who “get it.”
- Don’t try to convince the haters.
- As more people use it and it gets momentum, it will pull in more people naturally.
- It’s hard to doubt something that’s an obvious success.
- Coherence and entanglement are distinct.
- Coherence: if you trace it all back, everything will make sense together.
- Entanglement just means that if you pull a thread, it will pull on others.
- Things can be both entangled and incoherent.
- Under the streetlight of our attention, it has to be coherent to fit in our mental models.
- But in the fuzzy edges coherence matters much less.
- One of the reasons that dreams seem to make sense, until you start pulling on threads and realize how little sense the situation makes.
- A belief system is a tangled web of mutually supporting beliefs and evidence.
- Like if you said "drop the belief that you have a brother,” there would be any number of contingent facts that are entangled.
- The integrity of the tangled web has a viscosity of change.
- Humans entertain counterfactuals all the time, and we're good at it.
- Maybe we’re good at it because we're bad at tracing all of the implications and don't even see the impossibility of many changes.
- None of us is capable of understanding a thing that understands itself.
- This is one interpretation of an implication of Godel’s theorem.
- People will choose less friction every time, but no friction means no growth.
- A mentor helps put constructive friction in your path.
- That mentor might be a teacher, a coach, or an author.
- You have to trust the mentor that they’ll put useful friction in your way.
- They give you the hard truths that you’re willing to accept.
- You have to feel they are emotionally connected with your success.
- That is, the friction, as hard as it is, is going to help you become a person you’ll be glad to be.
- Making this decision is like doing a trust fall.
- A possible new equilibrium: security through diversity.
- Before infinite software, you might as well use the component that everyone else uses.
- A heavily used component will have attracted bug fixes from a larger catchment basin, and if you’re a less important target than other entities using it, you hide in their “rain shadow.”
- But now, every piece of software has bugs that can be found by burning tokens.
- Heavily used bits of software are more likely to have tokens blasted at them.
- If you have your own bespoke system, then the likelihood of tokens being blasted at it is tied entirely to how valuable of a target you are.
- As a teenager, when your parent starts using the same social network, it’s indescribably lame.
- When your parent joined Facebook was worse than when they started using Discord.
- Discord at least is sliced up into different silos.
- You’re less likely to see them or have them see you in your normal use.
- The modern world produces infinite shit.
- It’s hyper efficient at creating totally hollow things.
- An Aristotelian taxonomy of friendship:
- Useful: Circumstantial relationship.
- Pleasant: Shared passion.
- Virtue: Mutual admiration.
- Imagine a world without public-key cryptography.
- Distributing the symmetric keys and keeping them secret consumes massive amounts of effort.
- It would be centralizing.
- Instead of privacy being a thing two parties can manufacture between themselves, it’s something that requires coordinating with a mutually trusted intermediary.
- Communities with significant energy tend to have a clear boundary for membership.
- The membership helps keep the “warp core” at the core of the community under compression, so it stays ignited.
- Staying ignited requires staying above a critical mass of energy.
- There's a difference between prediction and explanation.
- An explanation can generalize to other domains better.
- A high-leverage model explains lots of outputs for a very small amount of moving pieces.
- Mathematicians like elegance for good reason.
- The three rules of bird flocks is much more interesting than the precise positions of every starling in a particular murmuration.
- When all of the complexity snaps into focus because you find the small model that generates it, is an aha moment.
- Inherently enjoyable.
- When you’re busy, you can take on more than if you’re not.
- When you’re busy, you can jump from task to task and stay focused.
- Your attention being taut helps allow laminar flow.
- When you have too much slack, you have turbulent flow.
- You spin up and then spin down in between.
- Getting momentum from a stand still is way harder than keeping momentum.
- Static friction is higher than moving friction.
- Different media allow different kinds of narratives and experiences.
- The narrative has to flow with the grain of the medium.
- If you film a stage show, it will feel off.
- If you approach a murder mystery evening like a novel, it won’t work.
- In a novel, every story beat is laid out in a precise flow.
- In a murder mystery evening, different people will make different choices.
- If you try to take a novel and make it into a murder mystery evening, then the key narrative beats won’t happen as you intend.
- Instead, you have to design for emergence: enough structure that interesting and meaningful things can emerge, but that they converge by default to the core narrative.
- We're going to be in a new golden age of software, but to get there this current dead and overgrown forest will have to burn down.
- When the world changes, everyone by default is still mourning for how it used to be and the opportunities that will never come back.
- The people who realize that the world has changed and can see the new opportunities that are now in the adjacent possible, and can do things that previously people thought were insane, but are now viable.
- What magic tricks can you do by realizing there is no spoon?
- Is the LLM architecture of fixed model weights fundamental or just an equilibrium around inference serving costs?
- Humans change our weights when we learn.
- But models are static and need a personal sidecar to save memories.
- Part of that is that models require huge amounts of capital to create and serve.
- The architecture of LLMs today is an all-powerful monolith, with a constellation of support structures.
- The long-term equilibrium will almost certainly be a constellation of different services, with different strengths and weaknesses, assembled into a configuration more powerful than the sum of its parts.
- Every unit of effort you spend papering over a fundamental problem has huge opportunity cost.
- The problem is fundamental and isn’t getting better.
- In fact, it’s likely getting worse as you pile more hacks on top.
- More and more epicycles crammed on top of the wrong paradigm.
- As you kick the can down the road, you delay the value that could be created by fixing it once and for all.
- Once you fix it, all of that maintenance effort to paper over it evaporates, and you can integrate that value over the future lifetime of the project: leverage that compounds.
- This compounding value is massive, but it’s hard to see because it is indirect and diffuse.
- It’s never urgent to plant a tree.
- When you see a familiar thing from a different angle,you can see its parts more clearly.
- If you can accumulate experience when others can’t, it’s a superpower.
- Explored in the movie Edge of Tomorrow.
- An example: as an adult teacher, the kids might learn robotics faster than you do.
- But each year is a new batch of students, whereas you as the teacher keep on accumulating knowhow year to year.
- The superpower is about the differential of experience acquisition compared to others.
- That differential is driven by how interesting your experiences are… but also by how good you are at extracting insights out of experiences.
- The latter is about curiosity.
- Complicated systems you can push to where you want.
- Just apply steady, consistent pressure, and they will move.
- Complex systems react to you.
- When you push, they push back and evolve.
- You can probe, and nudge, but no more.
- Things that are great have a compounding component that stands out prominent from the baseline.
- If you do a thing and everyone you show it to goes “What?? That’s crazy!!! … Show me more!” then just do more of that.
You can continue reading (there are thousands and thousands of pages!) in the backlog at [Archive 3] komoroske.com/bits-and-bobs. Every so often I have to create a new Google Doc because I hit the length limit!
[a]TBH, I don't agree with this.
Building great software is still pretty tough. Having a design opinion, building for high reliability, low latency, working across devices, handling permissions and what not. People shouldn't be building their own software for everything. I think we quite underestimate how much effort it still takes to build and run something.
[b]You barely give any weight to the benefit of ownership.
[c]I prefer to think of it as "earned autonomy" and the root of that trust is evidence that has earned the trust humans can uniquely lend. But can I rely on your trust? or does it have to be yours only?
[d]https://scalingtrust.com/
[e]Are you sure this is right? see here re creative insecurity https://books.google.com.au/books/about/The_Politics_of_Innovation.html?id=lkcnDAAAQBAJ&source=kp_book_description&redir_esc=y