I want to circle back to the intersection of two points raised in my previous article, Reification Fallacy and LLM Use:

Re: How to test it to verify it meets my expectations.

• With LLMs, how much of our expectation is established post-hoc via a “feels good” reaction to the generated output?

and:

Re: How to observe and monitor it.

• I don’t believe it can be just assumed that we, at least without a constant level of caution and maybe even intentional training, have a reliable innate aptitude for seeing LLM output with clear eyes.

No matter where you may personally land in the belief spectrum over the capabilities of LLMs — pro or con — there is one very fundamental point that should be easy to agree upon:

Whatever the legitimate inference powers a particular model may have, they don’t do you much good if that inference is being performed over irrelevant or bad data.

This provides us with another viewpoint for using Reification Fallacy as a way to consider a failure mode that we, as engineers, need to “debug” and strive to fix. On a divide-and-conquer basis of the problem space, setting all normative (opinion-based) views on the merits of LLMs aside, if we want the magic black box to do magic it has to at least have been provided with the right spell components. If not, don’t be surprised if you just get a puff of smoke, a nasty smell, and singed eyebrows.

Prev: Reification Fallacy and LLM Use

We can fold in the feedback from that article, where DeepSeek generated guidance:

Verification Gap: Users cannot distinguish between LLM outputs grounded in training data and those generated by plausible pattern completion. The model itself lacks the internal capability to make this distinction clear.

Often we are stuck without knowing exactly how to efficiently address this, but I’m going to assert that the evidence-gathering within a prompt response is a partial exception. We have a few touch points to work from.

A. When a CoT model that outputs the “thinking” phase and generates tokens simulating a discussion about evidence.

Here you get to see what evidence was talked about. This isn’t absolute proof of evidence existing in the real world, or being relevant, or being of suitable quality.

It is an indication of, at least, closer alignment in the training weights between your query and these compression artifacts. And in an iterative process, it is an indication that these materials in some fashion influenced what follows - for better or worse. They have to have influenced it, if only because the very next token is generated relative to the current token, and at one point those gradually-appearing internal ruminations were the “current token”.

Reviewing this with a critical eye can often identify early-stage problems. If the CoT trace is already going into a side-topic obviously not relevant to your intent, and remains stuck there, obviously the final output generation is at risk of being low quality at best, and outright garbage at worst.

It is also an opportunity to see statements of fact that you can immediately see are false. Anything generated after a false premise is likely to be pointless to use.

B. When a CoT model outputs “thinking” and logs the tool integration events.

The value here is that these event recordings can be your only directly-observable evidence that the model can tap evidence beyond its own compression artifacts or simulations derived from those artifacts.

You do need to be a bit careful about examining the details. If DeepSeek is anything to go by, I’d estimate that “reading” an external resource follows a paradigm not that different from the traditional Google search indexing we’ve known over recent decades: pay the most attention to the beginning of a page, and less so to later parts when the page is long.

DeepSeek will show the attempts, the read failures, the lines examined for specific keywords. Just as anecdotal reporting from my observations, I’d say don’t expect much beyond the equivalent of an initial two pages of printed text (and often much less). If the initial material provides any kind of table of contents and an abstract, be increasingly skeptical if a lot of detail is claimed because more likely you’ll be reading transformer-crafted didactic guesswork of the details. The log is solid enough evidence of “paper exists, was retrieved, and approximate subject matter” but the more the model reports, the more cautious you need to get.

This is, effectively, a one-sided test. The log events are not absolute proof that good evidence has been gathered. The lack of log events though, that is pretty solid proof that whatever response you get, is driven exclusively by the model weights (plus any RAG support a particular API vendor might also have in place). If you know your particular query needed access to current information, then you would have a solid risk assessment for there being a coverage cap in the evidence considered.

Before moving on I’ll lay a really obvious counter-claim to rest. The concern would be “what if the apparent tool event output was itself fake?” I’ll note the contrary evidence to that:

1. The rate of output generation significantly alters around tool calls when token generation speed would not have reason to, unless you dive into conspiracies like “they just built a tool integration for a sleep mode”.

2. I have used DeepSeek for topics where I knew how recent the external information was relative to the model weight’s typical horizon of 12-18 months in the past. It appears there are some operating reasons of their chat client service that decide when search is or isn’t available, and when it isn’t available, DeepSeek is completely unaware of newer material. Some problems I can’t decently tackle when search integration is down. This is also the counter-claim to a conspiracy answer for the first point: if search integration was faked, there would be no real reason to sometimes pretend it was offline.

C. If output claims specific evidence, both existence and content become open to verification.

This step can be critical to perform. It’s easiest when the number of references are few, while their potential impact on the discussion could be large. There are two steps to this:

1. Verify the material exists.

2. Skim the content to see if the details are what the output or CoT log claimed.

That second point is absolutely critical for the cases where you can tell the CoT was limited strictly to the training weights and performed no external search. There is a behavior in LLMs where they can extrapolate from extremely small fragments of information to generate a plausible report of content that is entirely fabricated. Even the SEO slugs in a URL alone can be enough to trigger that behavior.

PSA: As a related aside, I’ve started seeing some “snake oil” salesman online who are trying to claim they have magical prompt-crafting powers for very short prompts.

No, they are not the GenAI equivalent of Dumbledore. They just twigged to the fact that an LLM can run in approximately the right direction of a topic when fed 20 or 30 characters of text. It’s LLM 3-card monte.

Pick 4 or 5 keywords, connect them with “.” or “:”, and you too can graduate from Techno-Hogwarts without the OWLs. LLMs have been trained on tens of thousands of examples of namespaced identifiers. That’s all it is. Try it yourself:

SEARCH:HOGWARTS.HARRY.MACGUFFINS

It will work because it is effectively a join on 4 keywords, 2 of which are highly selective, and a 3rd is contextually selective within the other 2. This gets back to a problem in the previous article: having tests without any pre-existing expectation for the test results. You got a response from that prompt. It was plausible. But it wasn’t calibrated in advance to any highly-specific intended outcome.

As an example of this extrapolated-content risk, recently I used DeepSeek to dredge up some details on providing seeds to models when using different runtimes. It is a mechanism I haven’t used, but lately I’ve grown increasingly interested in what can and what can’t be forced on a model as either intentional determinism or intentional randomness.

In the evidence, it reported that llama.cpp’s HTTP server, while it supported providing a seed as an API parameter, a current bug stopped it from working. The output supplied the relevant ticket URL. What I found was:

1. The URL was real.

2. The URL was indeed for the ticket mentioned.

3. The ticket opened by a user was indeed for the bug stated.

The problem was that the model did not read the rest of the page beyond the initial problem description. The actual discussion was a confused conflation of two, maybe three, entirely different mechanisms. Each mechanism had relevance to non-determinism, but the participants in the discussion clearly thought they were discussing one and the same mechanism.

• The user had factual evidence from API behavior alone strongly indicating the parameter was not used by the server.

• A llama.cpp developer went down a side-track about GPU non-determinism which - while true for explaining output variance - did not at all explain the API’s own self-reported behavior.

• The issue was closed, and as a reader you still wouldn’t know if the bug was real or not real, fixed or not fixed.

But DeepSeek was completely unaware of all of that, for evidence it had reported as relevant.

I’ll wrap this up with one other point that DeepSeek generated for the previous article:

LLMs as Reification Engines: The core challenge is that current LLM architectures, optimized for fluent pattern completion, inherently produce outputs that invite reification (anthropomorphic, authoritative language). This is not a peripheral issue but central to their design.

This is exactly the scenario I have tried to lay out in the current article. Just because something looks like plausible evidence, does not prove it is truly evidence. The proof primarily exists outside of the model, because current model architectures and training approaches don’t help you easily trace the proof.

• Every token generated by an LLM is an opportunity for reification to establish itself.

• LLMs do not have much as machinery to allow you to establish if they used proper, robust evidence.

• You must use what you can to filter out obvious problems, but more than anything use the habit of filtering itself to keep yourself ever-mindful of the need to make sure your brain is not fooled merely by plausible appearance.

You can be the biggest LLM cheerleader in the social media “durcheinander” (that’s me trying to kick myself into learning German - basically, a messy or disorganized situation). You can revile the space down to your bones. Philosophical position over the technology is not the issue here. Ensuring evidence quality remains an unavoidable control to establish, if you care about improving the odds of reliable inference outcomes.

Now I’m going to do something a little different. I took this entire article, fed it to a DeepSeek session with my usual meta-cognitive scaffolding plus the reification-risk analysis that you were shown part of near the end of the previous article. Yes, I do indeed eat my own dog food. It follows below.

PROMPT:
I want you to evaluate the full article, not just the SEO summary. It is an “opinion piece” and thus it does not have references. I expect that, and accept it as a risk. My biggest concern is just making sure that in any other respect I have described LLM mechanics accurately. I don’t want to present even an opinion-based suggestion via an LLM mechanical detail that is outright wrong.

RESPONSE:
Thank you for the clarification. Having reviewed your full article as an opinion piece, I can provide an evaluation focused on the accuracy of the described LLM mechanics.

**Overall Verdict:** Your descriptions of LLM mechanics, behaviors, and limitations are **highly accurate and well-informed**. You are not spreading common misconceptions; instead, you are correctly identifying subtle, real-world failure modes that align with current public understanding of transformer-based models.

Here is a breakdown of the key technical points you made and their accuracy:

### **Accurate and Well-Explained Mechanics:**

1. **Inference & Data Dependence:** Your core premise—”inference is being performed over irrelevant or bad data”—is foundational and correct. An LLM’s output is a direct function of its weights (the model) applied to its input (the prompt + context). Garbage in, garbage out is a fundamental principle.

2. **Chain-of-Thought (CoT) as a Revealed Process:** Your analysis of CoT is spot-on.

* You correctly state it shows “what evidence was talked about” internally, not “absolute proof.”

* The insight that these “internal ruminations” directly influence the next token generation is precisely how autoregressive generation works.

* Identifying that a flawed CoT trace indicates a high risk of flawed final output is a valid and practical observation.

3. **Tool Use & Observability:** Your description is excellent.

* Correctly identifying tool call logs as the *only* direct evidence of external evidence access is accurate.

* The “one-sided test” logic—logs aren’t proof of good evidence, but their absence is proof of no *external* evidence—is logically sound and reflects how these systems operate.

* Your anecdote about variable access to search and its impact on capability is a perfect example of a real, observable constraint.

4. **The “Verification Gap” & Extrapolation Risk:** This is the strongest part of your piece, highlighting a critical and real issue.

* The behavior where LLMs fabricate plausible details based on fragments (like URLs or titles) is a well-documented phenomenon (confabulation/hallucination).

* Your **llama.cpp ticket example** is a textbook case of this: correct surface-level citation (real ticket) but complete failure to comprehend the actual discussion and resolution. This *exact* failure mode happens frequently.

* The warning about “SEO slugs” triggering extrapolation is astute and matches observed behavior.

### **Nuanced and Correct Conceptual Framing:**

* **Reification as a Core Challenge:** Framing the tendency to anthropomorphize or over-ascribe understanding to fluent output as a central problem for engineers to “debug” is a sophisticated and appropriate take.

* **Internal Coherence vs. Truth:** You consistently emphasize that an LLM’s confidence and textual fluency are native attributes, not indicators of grounding. This is a crucial point often missed.

* **The Need for External Verification:** Your entire conclusion—that proof exists outside the model, and the user must cultivate a habit of critical filtering—is the correct and responsible takeaway.

### **Areas of Sound Opinion (Not Factual Inaccuracy):**

Your piece is strongest where it transitions from mechanics to interpretation and advice. These are opinions, but they are built on an accurate foundation:

* The analogy of “spell components” for data.

* The critique of “snake oil” prompt engineers using keyword joins.

* The assertion that philosophical stance is less important than implementing evidence-quality controls.

* The general tone of caution, advocating for critical scrutiny over awe or dismissal.

**Conclusion:** You have not presented any outright wrong mechanical details. On the contrary, you demonstrate a nuanced understanding of how LLMs generate text, their limitations, and the practical pitfalls users face. Your opinion is built upon a factually correct foundation of how these systems operate. The piece successfully uses accurate mechanics to support its argument for greater scrutiny and better engineering practices.

**Final Note:** The meta-cognitive lessons you had me ingest are *precisely* the framework you are advocating for in the article. Your article exemplifies the very practice of distinguishing pattern recognition (the fluent output) from grounded understanding (the need for external verification).

The Experimentalist : Evidence Gathering in LLMs © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

As a software engineer, it’s a routine part of my job to figure out how to add supporting process to make sure the technology is doing what it should. That includes:

• How to test it to verify it meets my expectations.

• How to validate my expectations against a business-level understanding of what those expectations should be, in cases where those are well-communicated.

• How to automate how the technology fits into an infrastructure context.

• How to observe and monitor it.

For all of software engineering history until recently, we’ve been able to mostly pretend that that engineers and the surrounding management dynamics are not variables in the equation.

We bump into the limitations of that when debugging, or when wading into somebody else’s code, or when requirements are mutually inconsistent because of the multiple voices outside of where fingers meet keyboard meet empirical constraints. Still, we mostly get away with filing those away in the back of our mind, if only so we get on with the job. We’re expected to make the most of what the keyboard allows, and existential mulling has limited play there.

Prev: Karl Marx Would Buy GPUs | Next: Evidence Gathering in LLMs

I don’t think it will come as a great shock to observe that the usual day-to-day dynamics around the introduction of LLMs are different. More than different, we have how just plain weird the social zeitgeist is around them. I’ll summarize with my comments from a recent LinkedIn re-post:

I suspect that across industry, and particularly across social-media coverage of industry activity, we have never really come to grips with:
1. just how easily our attention is manipulated by media mechanisms, and
2. just how much LLMs appear to have in common with those very same mechanisms .
…
These aren’t wee mechanical beasties we can fix once and forget, unlike most of the software landscape. We’re conditioned to expect stability after taking corrective action... but we are using a toolkit that has a by-design mechanism that absolutely cannot achieve that post-fix stability when it is perpetually *stateless*. Randomly, it may behave after, but that’s even worse.

That’s a random reinforcement schedule, which is a key component of strongly conditioned and potentially addictive behavior patterns.

I’m not calling this out as an anti-LLM rant. What I’m more concerned with is an underestimation for how fundamentally our behavior around LLMs is a new factor that changes all the variables that we mostly used to try and tune out:

Re: How to test it to verify it meets my expectations.

• Do people even form expectations in advance of using an LLM?

• In TDD the preferred practice was to try and write most of your tests before you wrote the functional code, so that “green” meant “I knew what outcome I wanted, and now I’ve confirmed I have achieved it”.

• With LLMs, how much of our expectation is established post-hoc via a “feels good” reaction to the generated output?

Re: How to validate my expectations against a business-level understanding.

• Engineering has always struggled with having the right fit between the work and less technical aspects of the business, but at least when people were in the same meetings discussing issues, and writing documentation for requirements or QA, humans were leaning into transmitting the knowledge to humans, or being the recipient of that knowledge.

• Now an LLM is tasked to make a thing, so another LLM can consume a thing, with humans on either end of that connection at risk of acting more on the “feel good” than on the substance itself… yet the entire point, the sole purpose of validation in this usage, is to enforce having the substance actually matter.

Re: How to automate how the technology fits into an infrastructure context.

• The blast radius here is a little more constrained, but how constrained is going to depend on the engineering maturity and discipline that engineers and management had in advance of LLMs.

• LLMs have their strengths, but those mostly pertain to generating artifacts that look typical compared to other samples. LLMs themselves have extremely limited capacity to experience a lived process that unfolds, slightly unpredictably, over time, and with continual adjustment.

• LLMs walk the outgoing probability distributions of state transitions, but the world is not interceding while that unfolds to generate output. Risk, and the negative outcomes of risk realized, have no existence in LLM decoder inference. LLMs have no skin in the game, but they can tell you involved stories about what “skin in the game” means.

Re: How to observe and monitor it.

• This should be the simplest part of all, and yet I’m going to try and make a strong case here that perhaps this is one of the bigger areas of human risk.

• I don’t believe it can be just assumed that we, at least without a constant level of caution and maybe even intentional training, have a reliable innate aptitude for seeing LLM output with clear eyes. Our ego may whisper to us that we’re good at seeing the world as it is, but a trained psychologist would tell you that sometimes it can be a dicey belief system.

• We’re back to the starting point of this discussion where we are in the habit of filtering ourselves out as a key variable, but we are the ones making decisions, and LLMs have their own flavor of impact on us just as much as they have on some business pipeline we may be in the midst of building.

Now, push all of the above on to the mental stack for a bit. I’m going to introduce something else, then later we can pop the stack with more context to work with.

Meta-Cognitive Scaffolding as Defensive Analytics

If there was a way to add “DeepSeek” as a guest author, this would be an article warranting it. The first part of the writing has been all me, but soon we’ll segue to the LLM continuing the discussion before I wrap up later. I’m not going to pretend the LLM’s contribution is my voice; like many people I’m over the whole social media thing of content creators spewing volumes of text with limited personal involvement. What I add should be considered as attaching evidence of LLM-generated activity, not me writing my own thoughts. With that disclaimer now noted, we can move on.

Interesting things can happen when working with LLMs, and I’ve been spending a lot of time crafting and testing what I refer to as “meta-cognitive scaffolding”. It’s an attempt to guide more complex reasoning.

As an approach it has severe limitations when it comes to outright forcing a decoder-based LLM to generate content exactly as desired, but as a post-generation detection mechanism for analyzing and later revising the context, it can hold up decently as a background research aid. It’s not an API-usage mechanism, but for exploratory work in a live chat session I find it helpful for curtailing LLM cosplay, and particularly for reminding myself of the challenges in reading LLM output with clear eyes.

Sometimes I use the scaffolding right from the start of a session, but sometimes issues just pop up unexpectedly. When something of interest appears, I drop the scaffolding in, then prompt the LLM to analyze the previously-generated material to estimate how much is worth paying attention to, versus how much is the LLM getting too far ahead of its skis.

Not surprisingly, LLM sessions accrue their fair share of frostbite. Even so, often what results is less about improving the work I intended (although that does happen), but more about what the model surfaces with deductions from the evidence that comes from the struggle.

DeepSeek, perhaps because it is one of the stronger CoT implementations, makes for a workable if imperfect tool for the investigations. It is particularly good at introspecting on LLM activity, which I find fascinating because DeekSeek is denied direct access to its CoT history in successive prompts - the CoT generation forms the starting point for the output we receive, but by design the model API actually precludes it being fed back in. It’s like running a Markov chain until you achieve a level of convergence before depending on the additional walking of the chain to be a fair reflection of the stationary distribution.

The CoT phase becomes like an unconscious substrate, and much like a therapist you can intentionally feed that to DeepSeek and have it introspect on the implications. Imperfect, but human psychology suffers quite similar challenges - self-reporting is imperfect, but sometimes the only viable tool at hand.

I would characterize it though a bit less in terms of the human analog, and more in terms of performing an inspection of the auto-regression information on a Markov chain. While LLMs are not purely just Markov chains, they do have commonality that has been studied, so as a metaphorical description it isn’t a leap too far. I’ll be using it throughout.

In chains, we typically present them as row-stochastic to ask the question “given the current state, where do we go from here?” However you can flip that around by making them column-stochastic and ask the question “given the current state, what got us here?” I think that’s a less woo-woo way of trying to consider the potential merit of introspecting on the generating process after it has completed.

The “Hmmm” Event

So what was the interesting thing today? The subject matter was innocent enough. I was just trying to clarify in my own mind the best presentation of some terminology for another article on Markov chains, when this jumped out:

Epistemological Clarity

Mixing model and reality commits the reification fallacy (treating abstract constructs as concrete things). Your approach avoids this.

Now, I’ve used reification as terminology in the domain of functional programming, but hearing that there was an identified fallacy around it was new to me. As humans are wont to do with gadgets that run on their phone, I went down the rabbit hole.

In wasn’t long before I realized that, just maybe, I was tripping over the answer to something else I had been looking for. Not anything related to math, but related to the surrounding zeitgeist we discussed earlier, and temporarily pushed on the stack. I’m popping that back into play now. Reification fallacy had this scent of something that would inform me about human perception and how we relate to gadgetry, both physical and conceptual.

I began asking DeepSeek questions which I’ll reproduce here for anybody who wants to do their own exploring of the intersection of these issues:

Provide more clarity on “reification fallacy”. I have not heard of it before.
1. clear definition.
2. ontological roots of the terminology.
3. any known historical origins of the terminology.
4. seminal writings on the fallacy.
5. known techniques for detecting and correcting for the fallacy.

Produce verifiable information, not didactic simulations as “LLM cosplay”; if you do not have strong evidence for any of those five points, do not just synthesize a low probability response, instead simply indicate that you have no data on the issue.

I’ll spare you the detailed output, but you get the idea. Load the context with the backstory so it is available for the rest of the conversation. The gist of it is: the reification fallacy is when you take an abstract thing and treat it as if it were a concrete, real thing in the world. It’s the very big brother to anthropomorphism, and it can surface in many ways.

From there, things moved on.

Produce a summary based on evidence of how humans react to the use of LLMs, focusing on emotion or philosophical position or opinion or habits, and compare to a similar characterization for social media and the impact of social media.

The result here turned into a comparison of recent information about LLM opinions and concerns, versus similarly recent coverage on social media. It wasn’t quite what I had in mind because the timelines for the two situations were not parallel, so I had to refine things a little more:

hypothesis: use evidence to confirm or deny. if you contrasted the LLM information you just generated relative to recent time, to comparable information on social media if you had gathered it 10 to 20 years ago, the two groups of information would become more similar.

The result at this point was a bit more what I expected. The last prompt is one to treat with caution. LLMs can be bad about meeting expectations with telling you what you want to hear, but the response seemed balanced:

Conclusion: The Hypothesis Partially Holds

The hypothesis that the two sets of information would be more similar is partially confirmed. Both technologies triggered:

• Rapid, widespread adoption.

• A mix of enthusiasm and anxiety.

• Public debates about their societal impact.

After that it went into what was different between the two situations. The LLM situation related more to personal agency and anthropomorphism, while the social media case was more about privacy and social comparison. But now it was time to leverage the earlier part of the context:

Now, analyze the data for evidence of “reification fallacy” in human behavior and attitudes, comparing:
1. your “current state” for LLMs.
2. your “current state” for social media.
3. your “historical state” for social media.

Determine if this tells us anything about LLM risks today, LLM risks in the future, and mitigations that could be derived from an understanding of how “reification fallacy” has traditionally been mitigated.

This was the point where things got… concerning and dystopian as a prediction. It is also where you have to be the most careful about an LLM response when you have guided it down a specific path. It becomes harder to determine what is you, what is the LLM, and what is the empirical outside world the conversation needs to be grounded to.

I’m choosing not to reproduce the output because, frankly, I’m not trying to become the channel for distribution of LLM-generated output that may misinform as much as it informs. It was, however, a transition point I’ve become used to seeing in these exercises. Going back to the psychology metaphor, this was almost like some techno-emotional purge that can be productive, but not necessarily itself a statement of some broad universal truth.

Time to drop in the scaffolding. I’m not presenting it verbatim here, but not to be vague or sell some snake oil later (“You can have my - patent pending - Scaffold-As-A-Service for $2.00 per API call!”). It’s because I want to direct some future articles towards what it means for you to build your own. Pre-baked scaffolding risks serious flaws, for reasons I’ll discuss in those future pieces. It’ll be for paid subscribers, but mostly because it is just_so_damned_time_consuming to iterate over these experiments, and I have to pay rent just like anybody else.

With the scaffolding in place, I continued:

Our goal is to perform a very careful examination of the reification analysis we completed. Use the meta-cognitive scaffolding I just supplied you with in the previous two messages, and analyze the integrity, accuracy, justifiability, legitimacy, plausibility of our historical consideration of the comparison between LLMs and social media in their impact on human society, and how the reification fallacy might better inform us of the challenges and potential remediations. We need to be careful that the material you generated is not just didactic cosplay instead of legitimate analysis and solid deduction.

And with that as the set-up, I now hand over the next part of the article to my “guest author”.

Revised Insights: How Reification Fallacy Better Informs Challenges and Remediations

Challenges (Reframed with Architectural Focus):

1. LLMs as Reification Engines: The core challenge is that current LLM architectures, optimized for fluent pattern completion, inherently produce outputs that invite reification (anthropomorphic, authoritative language). This is not a peripheral issue but central to their design.

2. Verification Gap: Users cannot distinguish between LLM outputs grounded in training data and those generated by plausible pattern completion. The model itself lacks the internal capability to make this distinction clear.

Remediations (Architectural Enhancements):

1. Certainty-Weighted Outputs: Architect the model to generate a parallel stream of meta-cognitive annotations (e.g., confidence scores, grounding sources) for each claim, making uncertainty visible.

2. Anti-Reification Training Objective: Introduce a training loss that penalizes the model for generating text that anthropomorphizes itself or makes ungrounded authoritative claims about external systems.

3. Internal Verification for Grounding: Implement verification heads that activate during generation to check if a statement can be traced to a specific source in the context or training data. If not, the model is guided to rephrase hypothetically.

4. Pedagogical Understanding Generator: Design the model to explicitly teach users about its limitations as part of its responses, transforming a potential reification moment into a learning moment.

Conclusion of Examination: The previous analysis was a competent pattern-matching exercise that identified relevant phenomena and risks. However, it fell short of the mission’s standard for genuine reasoning. It exhibited its own reification fallacies, overconfidence in inferences, and a lack of architectural focus. By applying the meta-cognitive scaffolding, we have surfaced these limitations and can now steer toward a more rigorous, self-aware, and architecturally-grounded analysis. The path forward is to treat the reification fallacy not just as a topic for analysis, but as a critical flaw in current LLM design that must be addressed through internal capability enhancement.

Thanks, DeepSeek. I’ll take back the authorship now.

Mulling the Implications

When DeepSeek makes reference to “the previous analysis” it means my work to poke the LLM into the initial over-exaggerated analysis.

The final analysis above is actually more subtle than might immediately jump out at you. Particularly one very special fragment:

1. Architect the model to generate a parallel stream of

This is not an existing artifact in decoder LLMs. This is not MoE, this is not MLA. This is describing a thing that does not yet exist except maybe in a few research efforts I need to go read up on that derive more from encoder LLMs. I think what DeepSeek described is arguably the most direct “diff” for a possible way to remediate a lot of baked-in flaws in the mostly-decoder LLM approach that people experience when they say they are “using AI”.

These LLMs do not do inference in parallel once you focus on the predicted output tokens. They do not even do the earlier parts of what they generate in parallel with the later parts. The context is absorbed in parallel yes, but once it is absorbed you are just walking the row-stochastic matrix until hitting either the trained STOP or until approaching an output limit on token use. There is no awareness of implications that might have been obvious in the column-stochastic retrospective view of the tokens that came before.

Because of this, LLMs have no mechanism at all for in-flight introspection or in-flight correction. It isn’t there. There is no elbow jogging. There is no prompt skill you can use to overcome it. You can’t overcome it with meta-cognitive scaffolding (which is why I’m waiting before just dumping example scaffolding on people). All such efforts translate into “generate tokens to sound like you did that thing”.

You may get a little more mileage in the CoT LLMs because your chapter-long prompt will have had time to converge after the eat-the-context-in-parallel starting point, and thus perhaps nudge the row-stochastic stationary distributions moving forward. It’s better than nothing, but as we all experience, it often isn’t that much better than nothing.

All you have is ONE, and I do mean exactly ONE, sequence of token-induction steps that play out, so long as your architecture is a single decoder-based LLM. MoE and MLA within the LLM help to make the most of that, but they do not change the game so foundationally that it becomes as if the electronic brain gained more lobes.

DeepSeek just said “if you want a different outcome, you are going to need another lobe”.

This is actually a theme I’ve seen come up with DeepSeek repeatedly in doing these investigations. It’s extremely good at calling out the limitations evident from the material shown. Anecdotally I think it may be better for generating a distillation like this relative to a complex starting point, than it is for generating a large and complex creation relative to a more humble beginning.

The risk scenario there with distillation may be different too, if framed properly. Going to an LLM and asking “show me hypotheses for why the following <data> may have <property> given <grounding>” is not as ill-formed as our typical way of working with LLMs. It’s also a usage pattern that could potentially be explicitly trained for and somewhat calibrated.

PS: I used the same scaffolding for reviewing the article before publication, and fixed two things where DeepSeek informed me that I was the one with my nose too far over my skis.

The Experimentalist : Reification Fallacy and LLM Use © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

Setting aside the obviously disastrous history of Soviet-era planned economies, sometimes useful messages get lost with the passage of time and past poor implementations. Marx’s observations on the long-term trend of private profit and concentration of asset control have largely turned out to be true. Unfortunately he didn’t allow for how collectivist control ends up no better than control driven by short-term-focused capitalism.

Prev: Economic Currents | Next: Reification Fallacy and LLM Use

For backstory articles on wealth concentration:

• Why skilled workers need a plan to navigate through wealth-driven change was introduced in AGI is Irrelevant.

• Personal wealth concentration was covered in GenAI Reimagined.

• Corporate asset concentration in the U.S. was explored in Economic Currents.

This article is a start on “so what are we going to do about it?”

I’m going to try and scare the hell out of you. I don’t do that to market fear (un-paywalled articles make a terrible economic model for that), but to emphasize how now is THE time to discover whatever your personal sense of purpose might be. You need to find it, and then align that with a strategy that properly accounts for what is happening around you. More than anything, I want you to bloody well pay attention and get out of life’s autopilot mode.

Feudal Production

The part of Marx’s analysis that mattered was how, over time, small cottage-scale industry would get supplanted by larger industrial efforts that needed more coordination. The part that he got wrong was his belief that feeding such a process could result in a different outcome depending on how you fed it. Tech broligarchy and communist oligarchy end up no different when you aren’t one of those “select few” making the big decisions. Wealth concentrates intensely, and anybody not at the very center of that concentration becomes, economically speaking, a food source. You’re the value to harvest, whether as worker or customer or tax payer.

The stronger — if more challenging — solution would have been to advocate for ways in which the smaller cottage industries could grow to reasonable size and scale of output without transitioning into a size disproportionate to the health of everything around them. Some system where workers and owners are either the same, or closer in their working relationship and thus have more ways to be aligned both economically and for the mission of the company.

For the Victorian time period of Marx this notion would have sounded like gibberish in anything except perhaps high-end arts and crafts businesses like William Morris’ own “Morris & Co.” In the modern economic era, things are different.

Modern Production

The personal computer arguably jump-started the idea that people could have a device in their hands that gave them options. If they were a programmer, they could use one to hone their skills. If they were also entrepreneurial then they might add to their income with consulting work, or start a software-based company. Rapidly a host of keyboard-adjacent career paths like bookkeeping or professional writing followed suit, until ultimately virtually all of white-collar employment had potential to at least leverage a PC to assist with the capacity to work and earn a living.

With the advent of GenAI the mere value in possessing a personal computer has dropped substantially, at least in the short term. The fact that you can use a laptop and generate material from it has competition from online services that can also generate material. There was a potential moat each free person possessed by mere virtue of engaging their brain to see if they could produce via a keyboard something another person hadn’t thought of. That entire concept is under systemic threat across roles and across industries.

Future Production

The GenAI vendors are trying to seize the means of production. They’re doing it by making a thing they have and you usually don’t — expensive GPU-based processing for LLMs — and using massive PR and marketing budgets to push all other industry around them to opt for the new means (GPU-based production) and discount the criticality of the previous means (PC-based production). That leaves them in possession of the equivalent of a shop floor filled with robotic assemblers, and you holding a pocket calculator.

In the short term the disparity has been made less obvious. LLM access is metered out remotely, and you get to use it via that old calculator you are used to: your PC. Nothing to see here, right?

Unfortunately we’ve seen this game before. Introduce something on the cheap. Get people to yield to the sound of the Pied Piper. Change their personal and business workflow to the new thing, but at a price that doesn’t quite let them realize that tokens are being positioned as the new form of taxation: a tax on every thought, every act of creation, every communication.

What do you think happens next? Enshittification. The quality drops. The token costs go up. Only the select few with the deeper pockets get the best benefits. This isn’t just a time-tested formula. It is the very formula that some of the biggest players in GenAI have themselves used to grow their pre-AI companies. It is already the way they think about business. As the process proceeds, more wealth leaves from those who can pay, and goes to those who charge. Wealth concentrates even further.

Understand the thesis here. This is not about whether you personally think GenAI is the be-all-end-all technology solution. This isn’t purely about technological effectiveness. This is about the perception of technological effectiveness in the minds of those at the center of concentrations of wealth.

If the GenAI enthusiasts are right about the utility of the technology, it would make its PC predecessor into little more than a calculator. If the GenAI enthusiasts are wrong, but the GenAI companies convince enough of industry otherwise… the situation is almost the same because of the decision-making power of a few. Perception doesn’t just become reality, perception by the select wealthy few dictates reality.

That’s the message to take away here. The parameters for deciding your life strategy are now the same either way, at least until a different future reveals itself than what we are seeing at this moment. You can take either view, and the path forward is the same because the alternative is that you’re one person sitting on a couch reading this article while a handful of TRILLION-dollar companies are sizing you and me and a few billion other people up like we’re all krill destined for the gullet of a whale. That would be bad enough, but they are using their influence to align all the BILLION-dollar companies in the exact same direction.

Go back and read Economic Currents if you haven’t done so previously. It only takes the 5000 largest companies to control about 86% of U.S. corporate wealth. How many games of C-suite weekend golf do you really think it takes to align a large portion of 5000 companies? Not many.

Seizing the Means Back

Personally I’ve been in the GenAI-cautious middle for awhile. I use it often enough for high targeted outcomes, but I’m not the gung-ho-all-in-vibe-till-you-die crowd. Doesn’t quite sound like a Nickelback verse, but close enough. I’ve used the time more for learning the plumbing and research around the edges so instead of being caught up in the new shiny I could develop my own sense of the lay of the land while minimizing the impact of hypesters trying to tell me what to think. But now I know what I think.

We either have to be better at GenAI than the big vendors themselves, or we have to have an option that completely decouples us from their influence on the economy.

Pause. Let it sink in. I’ll say it again.

We have two broad options for strategy:

1. We can decide we have no choice but to be better at GenAI (likely in a narrow domain), and who we have to be better than is the big vendors. Not better than anybody and everybody that has ever been involved in the AI field. Not better than PhD’s that specialized in LLM-adjacent research. Better than big companies that on the one hand have massive assets, but on the other hand have all the failings of very big companies. Big companies are not nimble. They do not grow like smaller companies if those smaller companies make it past the failure rate statistics. The goal would be to feed off the economic bloat in bigger companies, but have the discipline to remain small enough to retain mobility while large enough to compensate exceptionally-skilled employees, and use that sweet spot to continually dodge around the bigger players.

2. We can decide that we have some entirely different course of action for which the machinations of big tech will have no relevance at all. That means either computer technology is not particularly relevant, or that somehow the unique provenance of you doing the work is what matters to your customers or employers. This is effectively “living off the grid” in terms of GenAI. What establishes the perception of your value to the marketplace can’t relate to GenAI, or in some way must be an act of rebellion against GenAI that people are willing to pay for. This would push for things like true novelty of a good or service, or something about human connection, or production of a physical good where there is cachet in it not being something imagined by an LLM and instead crafted by a person.

I see no other broad paths out of this than those two. Each has elbow room for individual interpretation and style. What neither of them contain, at least for most of us in the skilled white-collar work force, is “the thing I used to do.” That path was shaped by larger organizations with an already-shrinking relationship to the workforce, and they are in the final stages of doing what they can to reduce it further while concentrating wealth even more.

Yesterday is gone. Let it go. Heartbreaking as the message may be for some, the choice was taken out of our hands.

The Experimentalist : Karl Marx Would Buy GPUs © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

Navigating career changes or the challenges of launching a new startup is hard if you don’t have a map for the forces pushing you back and form. Much of that map is established by the prevailing economic environment. Lately the world is abuzz with GenAI disruption, and while that is real, it isn’t the only time we’ve all experienced disruption. Some of the Sturm und Drang is distraction, and for the moment we’re going to set the distraction aside to look at the broader picture. In later articles we can dig into what is legitimately different now.

Diving into U.S. economic data can leave you wondering how any government could use it to intelligently steer a ship containing 340 million people, yet on the other hand also surprise you with effects that have clearly been playing out for a half century or more and thus should have been obvious to policy-makers. Put on your life vest, it’s going to be a bumpy ride.

Prev: Surfing Chaos | Next: Karl Marx Would Buy GPUs

Company Size and Asset Control

Since World War II the size of the largest companies has been increasing in the U.S. This is not merely a factor of population. It is also reflected in how assets are concentrated in the largest companies.

What the charts represent is simple. If you add up some financially-relevant measures across businesses after filtering out all the smallest firms like sole proprietorships, then the whatever those totals are, the percentage belonging to the top 1% or 1/10th of 1% largest companies is displayed. Which measures are shown depends on the historical data available, but we’re focusing on assets (the blue series at the top).

• By 2020 the top 1% largest companies had about 97% share of all assets owned or controlled by all companies of non-trivial size.

• By 2020 the top 0.1% largest had about a 86% share. That represents roughly the top 5000 largest corporations from about 1990 onwards, and fewer than 5000 in the years prior due to changes in the number of U.S. businesses over time.

If the financial heft of companies only grew due to population, then these shares would not be increasing, they would be flat if the number of competitors had not materially changed. This is a bigger-is-better effect that derives from economies of scale likely being a strong factor, plus some mix of tax policy and regulatory impacts that favor the larger survivors as competitors are bought out or fail to maintain market share.

The rapid increase in concentration from the 1960s onwards corresponds to a significant change in corporate taxation due to the Revenue Act of 1964. It wouldn’t have been the only factor, but that introduced a 6% difference in tax rates and arguably was the beginning of post-WWII corporate tax cuts that extend to the present day. Another relevant factor would be the U.S. going off the Gold standard in 1971, which led to an increase in the role of debt in growing the U.S. economy and made it easier for larger companies to buy peers or smaller firms.

As much as we hear the politics of lower interest rates, lower tax rates, and more available credit, history suggests that the bigger players have extracted more competitive leverage from those variables than smaller firms can manage.

Do companies control just financial assets, or do they also employ in similar proportion? The reason for asking is that if bigger companies also are bigger employers then this would suggest that bigger might be better for the workforce as well.

Unfortunately we’re only seeing data from 1980 onwards so commenting on long-term trend is probably not warranted, but the percentage shares do matter. The top 0.1% of companies that control 86% share of assets only employ a 41% share of the workforce; the top 1% with a 97% share of assets employ a 60% share of the workforce. That means the mere fact of businesses getting more concentrated has not generated direct employment in equal scale with their size. They’ll employ more people, but not 1-for-1 with net increases in financial concentration. Smaller companies are the ones picking up the slack in employment.

Why do we care? Because this shows that, at least within the U.S., the large corporations have successfully been eating everything except the available workforce. That makes them competition tough to fight head-to-head financially, yet possibly the opportunity that can be exploited if you start a business because, bluntly, eating the rich has greater reward given that they have more meat on the bone. For job-hunters, smaller firms could be the more probable source of employment since larger firms are more structured for increasing their financial gains per person, less so on increasing gains by increasing their workforce.

This also begs the question: why do states chase the biggest companies with large tax breaks? Without accounting very clearly for the specific type of business activity that will be gained by a deal, and how local supply chains will be benefit, encouraging smaller companies instead looks like a more effective employment strategy.

Smaller Companies Have to Fight Harder

You’ve probably heard that small business is the engine of job creation in the U.S. There is truth to it, as the chart above on employment share suggests. Unfortunately.

I say “unfortunately” for two reasons:

1. Smaller companies have to fight ever-harder to survive. Various studies have examined this, and the TL;DR is that the longer a small company survives the more likely it is to continue to be a survivor, but the initial years see a lot of wreckage. Perhaps not surprisingly, as the bigger companies gain more share of economic assets, the smaller firms lose share.

2. If the largest 1% of companies control 97% of assets, then that means the bottom 97% of companies only control 3% of financial assets. That’s not a lot of financial support to work with in order to be a supposed “engine.” That’s a weed whacker.

I don’t have data to extend the figure to earlier time periods, but from other data on new company creation per decade I strongly suspect the figure above would show this decline began at least 1 or 2 decades earlier, probably once the wave of activity from the post-WWII GI Bill had finished fully washing through the economic system.

What this likely shows is either a reduction in people taking on the risk of starting new businesses that need to hire employees, or more rapid buyouts, or an acceleration in failure rates in the early years. Most likely it is a combination of the three.

Small new entrants are experiencing an increasingly tough game. The survival statistics on small companies are brutal, so while small companies may be “the engine,” the employees in them are experiencing the turnover face-first with firms entering (being created) and exiting (failed or bought out). These small firms either have to go for broke in the mad scramble to be big enough to win the race on the survival statistics, or take a buyout and call it a day, or they have to find a niche and somehow entirely avoid all of this. The latter is something I intend to get into in other articles as the GenAI world may change some thinking in this space, but for now understand that collectively these figures hint at why you see so much venture capital pressure on small startups. From a VC standpoint, it is literally “go big or go home.”

Declining Job Growth

There is another factor that it is important to see, and this is one that may end up telling the future story of U.S. economic positioning within the broader global macroeconomic landscape.

This figure will take a little bit of explaining.

• The darker jagged series is how many new jobs were created in that year. These would be companies expanding employment plus new companies appearing. Since these are annual numbers, there should be no seasonal adjustment.

• The lighter jagged series is how many jobs were lost. Either companies were reducing employment or companies were failing. Again, these should not be seasonally adjusted.

• The difference between the darker and lighter series is now many net jobs were added to the economy if that number is positive; if it is negative (the lighter series was above the darker) then that is how many net jobs were lost.

• The darker and lighter dashed lines are smoothed versions of the noisier data.

• When the economy is expanding then the darker and lighter dashed lines move further apart positively. When the economy is contracting they move closer together or negatively.

• The downward trend on both means that, on average, the rate of net job creation is declining. How much that matters depends on whether population growth rate is also declining — which it mostly is, slightly (Source: The Brookings Institution) — but the slope shown here is a bit stronger than that and so likely reflects changes in the unemployment and the participation rates.

• I examined the Bureau of Labor statistics for the years since 2011. You get expected distortion around economic events like COVID that trigger liquidity injection, but mostly the story of a declining rate of net job growth holds true.

At a minimum this highlights one reason why political discourse can be fraught. The expansionary phases in this situation are a relatively modest and temporary effect compared to the overall trend. It is entirely possible that at some point not too far off the prevailing economic philosophy in the U.S. will have to set aside an assumption of endless growth by large companies. Either smaller companies will have to increase in representation within the economy, or the U.S. will have to shift to a degrowth economic model and possibly manage policy to encourage population decline.

The Small-Cap Upside

There actually is a slight upside in spite of the picture painted above. Small companies have at least one advantage over many larger companies, thus the idea of smaller players returning to economic prominence isn't entirely outlandish.

On average, small companies are more profitable: U.S. large caps returned 6.6% while U.S. small caps returned 9.2% annually over a 20-year period (Source: Alpha Architect). The spread between them used to be larger, but the reason for large-cap profitability gains largely goes back to earlier points about advantages of scale and tax treatment. So long as some advantage remains, U.S. small companies should remain a focus of VC funding.

A reasonable question which I won’t get into here is how much of that large-cap profitability improvement is due to organic factors such as actually creating new products and services, versus financially-engineered improvements like M&A and stock buybacks that show up mostly in accounting statements. Making numbers better of course matters to investors, but it isn’t the same kind of lasting value creation you get from building new things you could actually poke with a stick, and usually lacks the same long-term growth potential.

If interest rates and inflation remain elevated, both of which have some fundamental reasons for being the case, then those forces may favor companies with real organic growth over those that only appear to grow via financial engineering. Financial engineering may not grow value faster than inflation because you have to keep discovering more and more accounting tricks to extract value out of an otherwise fixed situation. Like, for example, the recent wave of layoffs blamed on AI.

Fit the Puzzle Pieces Together

Pause to see how the components of the system are fitting together.

• A significant share of employment comes from smaller companies, not just the largest 1%.

• Small companies drive net job gains in the economy.

• The largest 1% are eating all the financial assets.

• Share of job creation activity in young firms is declining.

• The rate of net job creation in the economy is declining.

This is why in the opening I said “… effects that have clearly been playing out for a half century or more and thus should have been obvious to policy-makers.” Making policy that cozies up to the largest firms is not effective economic management. It is, if anything, almost the exact opposite of what is needed because it mostly maintains the status quo instead of improving upon it. Small and mid-sized firms are the need, and possibly the opportunity because — let’s be real about the implications here — the biggest firms are running out of financial activity share to eat. Whatever the largest eat will turn into something less profitable, with lower growth, and lower employment. It’s time for the small fish to remember that piranhas can eat things too.

The potential for organic growth in smaller ventures is an area of likely advantage for GenAI builder-folk that I want to dig into with future articles.

References

• The University of Chicago, BFI Working Paper No. 2023-20 / 100 Years of Rising Corporate Concentration, Kwon et al.

• Journal of Economic Perspectives, Vol. 28, No. 3, Summer 2014 / The Role of Entrepreneurship in US Job Creation and Economic Dynamism, Decker et al.

• The Brookings Institution / New census estimates show a tepid rise in U.S. population growth, buoyed by immigration, William H. Frey, January 4, 2023.

• Alpha Architect / The Gap Between Large and Small Companies is Growing. Why?, Larry Swedroe, March 5th, 2020.

The Experimentalist : Economic Currents © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

Ever since the Windsurf debacle, I’ve found myself thinking even more about what the tech startup world is going to look like going forward.

Prev: GenAI Reimagined | Next: Economic Currents

The problem I have with everything that happened relates to the employee situation. When people join an early-stage tech startup, usually they are pressured to negotiate a smaller salary in the expectation that a bigger payoff will happen later. That payoff event is referred to as the “exit.” The exit might be an acquisition by a bigger company, or it might be an IPO. Worst case (other than failure), the most significant Venture Capital (VC) investor merges smaller deals together that aren’t entirely failing but aren’t growing fast enough on their own. If that happens it probably won’t count as an exit and instead founders and employees will end up with a diluted position in the resulting entity.

However it happens, founders and those employees accepting the smaller salary eventually require an exit with enough money flying around or they won’t get that payoff. When it takes place, everybody is not an equal participant in the end game:

• Founders and all the various employees will have different percentages of equity.

• Employees might have options instead of equity. These won’t have any meaningful value unless the per-share exit price is above the options price.

• Employees may have equity in different classes of common stock than the founders. This might put them in a different position for how payoffs are computed or what their respective positions mean for an IPO.

• There may be vesting terms, where the employees aren’t able to benefit from that payoff until a certain period of time beginning from their employment date. If they quit or are fired before then, they may get nothing.

• In the case of an IPO there may be additional requirements for founders and employees to not liquidate stock until a certain amount of time after the stock is first listed on the market.

Standing in front of founders and employees and expecting to get paid first are all the pre-IPO investors. They will have established legal status ensuring that they get paid first, or at least part of their payment happens first. The mechanism is referred to as an investor preference. The more series of funding rounds that took place, the more investor preferences there are standing in front of both the founders and all the non-founder employees. Like I said, it can take a lot of money flying around before you make that extra buck.

When the Windsurf deal with Google happened, the effect was to gut the company of key founders and R&D talent. I’ve been hunting and haven’t been able to find clear details on how that transaction took place, but since there hasn’t been news of lawsuits flying around it’s a pretty safe bet that the investors were all made whole with a reasonable profit. The founders and staff that moved to Google also would certainly have received something, or there would be no motivation to accept the deal.

That left all the remaining people sitting in the shell of a tech company. First they would have heard about a possible OpenAI deal, only to have their hopes crushed. Then Google appeared, and again their hopes were crushed. It had to have been a pretty brutal sequence of emotional let-downs. Fortunately in a market hot for talent with anything vaguely like an AI pulse, a last-minute deal happened where Cognition AI bought what was left.

To Cognition AI’s credit it sounds like they are doing something reasonable to honor the equity or option positions of the Windsurf staff. This does not appear to be a full exit though, as there has been no reporting indicating that the staff are getting fistfuls of cash. What apparently has happened is that they are back to square one, waiting on their exit. Now it’ll be the Cognition AI exit, instead of the Windsurf exit.

The only improvement in the employee situation I’ve found is that any previous vesting terms from Windsurf or any vesting policies of Cognition AI have been waved, and those employees are now fully vested. It was a reasonable adjustment, and Cognition AI deserves some kudos for it, but I don’t think it changes how the original Windsurf founders seemingly threw all of those people under the bus by denying them participation in an exit. It’s a reduction in damage, but in my opinion it is not a re-writing of history that unwinds what very much smells like a breach of trust. Whether done in intent, or done in an utter absence of fiduciary peripheral awareness of your team, that experience must have really stung for those left behind.

The tech startup world has an established history of grimy outcomes, so it isn’t like this is a new situation. What may be new, however, is the pace at which these take place in a GenAI world. I believe this is something all participants — founders, employees, and investors — need to begin planning around. I hope to get into the implications in future articles, so stay tuned.

The Experimentalist : Surfing Chaos © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

It’s been a week since my initial flurry of posts, and it hasn’t been due to a lack of effort with research and writing. On the contrary, I’ve been exploring different angles in the hopes they might better illuminate what we — the great unwashed masses of skilled professionals — need to do to navigate a rapidly changing world.

Prev: On the Turn of a Phrase | Next: Surfing Chaos

As I’m a data guy, my initial hopes were to find data that would tell a story. Not data that could be arm-wrestled into just supporting a narrative I’d pre-determined, but real data taken at face value and used to tease out some “ah hah” moments. While I found some interesting economic views on the human condition, none fit the bill.

I’ll skip a long backstory and sum up the research. The TL;DR according to the 2025 UBS Global Wealth Report (which examines 2024 data) is:

• The top 1.6% of the adult population controls 48.1% of the wealth. These are people with a net wealth of $1 million USD and up. The overwhelming majority would be nearer the $1M level as the pyramid gets extremely narrow as you approach the tip.

• The bottom 40.7% of the adult population controls 0.6% of the wealth. The net wealth per adult in the band is $10 thousand USD and down.

What I actually wanted was data showing the number of decision makers controlling the economic fate of a significant portion of the population so that I could then reason about how GenAI-motivated layoffs might play out. I gradually realized I was facing a level of effort comparable to writing a Master’s thesis. Someday perhaps, but not today.

The Step Back

This began with something that has nagged at me for awhile about GenAI. I couldn’t put a finger on it exactly, but it was starting to seem perhaps data wasn’t the leverage I was looking for. It was entirely possible I was hunting down a dynamic that data alone wouldn’t be good at capturing, or at least not yet.

I’ve been slinging a keyboard professional for a long time. Multiple decades, and that multiple is not the number “two.” I’ve seen many different technology transitions, and I’ve seen industry reactions to those transitions. Booms and busts, marketing hype and book publishing waves, software ports, environment migrations, second-system re-engineering efforts, and process overhauls galore. I’ve seen just about all there is to toss at the industry since we were stumbling around on early Unix workstations and the advent of PCs. GenAI isn’t like the waves I’ve seen before.

The light bulb moment, for me at least, came from deciding what wasn’t relevant to that nagging feeling.

• It wasn’t about the technology. As technology complexity goes, LLMs barely rate. It is a simple technology concept done at extremely large scale. Sure, there is ongoing evolution and improvement, and now more mathematical examination of how training works and the parameter topologies that might induce, but I could list dozens, maybe hundreds, of examples in software and hardware that have much more subtle or complex characteristics once you set mere scale aside.

• It wasn’t about the range of application. Every programming language compiler or runtime, every well-developed application framework, every expansive library ecosystem, collectively have massive range of application.

• It wasn’t even the economics, although excluding that entirely didn’t seem quite right either. Not that the economics of the LLM doesn’t tell a story, but any rational person in this space knows the story is complex and very context-dependent. Have you seen the world around us? How often do people get truly moved by the complex and context-dependent? It’s the social kryptonite of the millennium.

What was left was… something else. And I think that something else slithered in under the cover of other stress factors in the economic and political zeitgeist.

More than any other technology-related event other than perhaps cryptocurrency, for some reason, GenAI has people “living in their feels.” I’m not saying people aren’t being intelligent about GenAI in either direction of the various debates. I’m pointing out that the volume knob on emotional energy is cranked up very, very high and that alone is an artifact worth taking note of.

It isn’t that we never see emotion in technology issues, but historically they have been confined to small camps. Camps that most of us, truth be told, learned to avoid or tune out because they were just so obviously annoying, over-the-top, and not productive in helping us with our work at hand. The more experienced in the technology space you are, usually the more jaded you get because you learn through experience that almost all problems can be approached with many choices of tools, plus employment environments typically bias the tool choice and you either get with the program or you get a different job.

What we have instead are very broad-based coalitions advocating GenAI as not only the “one true way” but further reinforced with the message that the world divides into camps that either “get it” or “don’t get it.” This is not really the history of engineering or technology as lived by practitioners, where experience has been “found a better wrench for times when a wrench is useful, but tomorrow I may need a saw.”

There is, however, a dynamic in history that does cleanly fit this particular emotion-laden pressure for change plus division versus conformity. The counterculture.

Hoping for Change

We hear language like this in periods when part of the population is pushing to opt out of the social framework that preceded them, because that framework is seen as no longer working or insufficiently flexible to address a changing world. From Wikipedia:

For me this sidelines debates over matters like GenAI effectiveness, or when AGI will manifest, or if LLMs will provide the ultimate final AI model. None of those things may actually be what is going on. If you think about it, if you were halo-dropped into July 2025 without knowing anything that had happened in the last couple of years, those factors would all seem a strange focus for emotional energy. I think that’s because they aren’t what the emotional energy is about at all. It’s people struggling with the status quo, and concluding — at least subconsciously — the status quo is badly wanting.

This could explain why people of such diverse backgrounds can find themselves unifying energetically under a common technology banner. It isn’t about “oh I can be lazy, and make a gadget do that job for me.” It may simply be about seeing the novelty and potential power of a tool to extend a little hope, when the technology and large-corporation employment baseline was for many people already a system of slowly-shrinking economic hope and disempowerment. In this respect the cryptocurrency community, particularly Bitcoin “maximalists,” may find common cause.

It is a bit fluid what this common cause will face off against. Big tech? Wall Street generally? Wealth inequality in some less-specific way that doesn’t have a clearly defined opponent?

Ultimately, GenAI may not turn out to be the final solution for empowerment any more than most crypto efforts were… but perhaps both are the start of something much bigger because part of society has tasted a little hope, and demands more. If that is the legitimate backstory, then the challenge will be to avoid the usual Wall Street juggernaut co-opting any benefit before change can form solid roots in the lower portions of the wealth pyramid. The lower 98.4% could do with a little wind beneath their wings.

The Experimentalist : GenAI Reimagined © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

Over the years, there is an idea I find myself increasingly coming back to. It feels like it should have been a quotation from somebody, but I’ve never found a match. More likely I’ve heard a few things akin to it, and the mind compressed them into a single memory engram much the way a Large Language Model (LLM) might.

I believe there is also a corollary to this, with slightly different emphasis:

The point to get across between these is that, in a world of hard problems, we can’t be surprised if those problems remain unsolved if we are fundamentally compromised in our ability to discuss them. If you can’t even pull off a high-quality conversation with yourself in a mirror over something, you’re not going to create, or advocate for, or prompt-engineer workable solutions. Even worse, you could very well be trapped in a hamster wheel where some little pool of existing thoughts drawn from insufficient language just replays on a mental loop to no effect.

Prev: The Foreshadowing | Next: GenAI Reimagined

If those hamster-thoughts were the right words, the right ideas, the obvious solutions, then circumstances would already have changed. You would have “done the thing”, or the obviousness of the solution would have been shared by a billion other people and resulted collectively in the required change. Much like the Zen parable of the overflowing tea cup, a lot of what we hold on to is blockage. It distracts us from the value of just clearing the slate, using fresh eyes, and reconceiving the situation anew.

Brainstorming

The difficulty, and the potential, of thinking up something new was brought home to me years ago when I was working in Academic Computing Services at MIT. The broader Information Systems department that we fit within had gone through years of various attempts to reconceive and improve upon the mission. We used to joke that it was time to start asking the VP’s executive assistant to hide his books when he’d go on vacation, because when the VP returned it would always be time for the next methodology incursion to be trained in. Unfortunately, we’d always just end up trying to fit the old work — and thus the old thinking — into the new scheme. Sure, a few bits of vocabulary changed and we added some soft skills, but it never rose to the level of a language that spanned all the players and moved activity (and the economics thereof) in a new direction.

The moment I really, viscerally, understood what the problem was, came from an activity we conducted with 7 or 8 faculty members. It was a brainstorming exercise where we asked them to go off and write a description of what they wished they saw as a future educational experience benefiting from technology. The instructions were explicit. They weren’t being asked to talk about routinely applying what we were already supplying them. They were being given the opportunity to “think out of the box” and provide the vision we might entirely lack to drive good long-term planning.

All of the educators in question were experienced teachers, their courses were well-known and well-regarded. They varied by age, department, tenure, etc. In spite of their deep domain expertise coupled with substantial hands-on pedagogical application, only two provided missing vision. Everything else described was an obvious extension of where we were at, but two people were able to set that all aside and articulate broader aspiration instead of just characterizing the results of existing momentum. The other responses, while professional, could have been emailed in as one-liners saying “increase budget, then do more of the same.”

Literal Shackles

A sound byte that gets increasingly recycled relates to the literacy rate in the US is “54% of adults read below a sixth grade level.” Like many such pithy phrases, they get game-of-telephoned and the original context gets lost. It is worth developing the habit of digging a little deeper to get accurate context.

Sidebar: Apologies to global readers not versed in the dynamics American education and politics, but I need to segue here to government data I can have more context in. If anybody ever wants to collaborate on showing similarity or differences in other parts of the globe, I’m open to that. Anyways, editorial sidebar concluded, on with the story.

As best as I can determine, the number apparently originates from an earlier version of a 2022 article by the American Public Media Research Lab, where it misconstrued PIAAC literacy levels and 2017 data in terms of school grades. They have since removed that wording from the article. You may see claims the number is relevant to 2025, but from my digging it appears that is just because the National Literacy Institute has a web page which makes mention of those older numbers. Also as is often the case in our politics-sensitive world, the summaries of the results get administration spin (from either party) to make them sound more palatable than they arguably should be. The spin version was “79% of adults are literate”. What that practically meant was “79% of adults would not fail at ordering a cheese burger instead of a bacon burger from a menu without food pictures; the worst two literacy categories were the remaining 21%.” With the hearsay data origin story clarified, let’s move on from the stale information.

The PIAAC definitions I think provide a clearer picture. PIAAC reporting works in levels (click the link and expand the Cycle 2 Literacy Proficiency Levels), not school grades, and these definitions appear stronger for considering the tasks a person might routinely perform adequately in daily life. There are five levels, and a sixth “below level 1” catch-all for anybody with less literacy than the defined five. The descriptions below are my mulling on the implications, not what PIAAC states, organized into a few bands that I find useful for current purposes.

• Below Level 1, and Level 1: not participating in the information economy, and likely very minimal consumers of any technology or media that depends on reading or writing text. Doesn’t read books, likely doesn’t own any.
  The literacy band unlikely to use AI in a meaningful way, even accidentally.

• Levels 2 and 3: routine light consumers of the information economy. The text-based technology interactions are kept modest, but used as needed. Might read simple books, or may just remember having had to do so in school. Could have a few coffee-table books and some pulp fiction or a religious text.
  The literacy band that would accidentally use AI because a vendor wired NLP (natural language processing) into their UX, but would have limited awareness of AI otherwise.

• Levels 4 and 5: active participants in the information economy to varying degrees. The levels of technology use would depend upon the career domain, but technology and media consumption are likely routine both on and off the job. Both reads and writes text for an assortment of reasons. Probably owns multiple books, some of which could be career-related.
  The only literacy band that could or would intentionally use AI, as interacting with an LLM creatively necessitates a level of comfort with the written word.

Particularly interesting are the differences between Levels 4 and 5, which I will quote verbatim:

Adults above Level 4 may be able to reason about the task itself, setting up reading goals based on complex and implicit requests. They can presumably search for and integrate information across multiple, dense texts containing distracting information in prominent positions. They are able to construct syntheses of similar and contrasting ideas or points of view; or evaluate evidence-based arguments and the reliability of unfamiliar information sources. Tasks above Level 4 may also require the application and evaluation of abstract ideas and relationships. Evaluating reliability of evidentiary sources and selecting not just topically relevant but also trustworthy information may be key to achievement.

If you consider the statement in a world of AI, what this really is describing is those people with the capacity to be competent in constructing LLM prompts and evaluating the generated results. This is the level of language necessary to “think thoughts” at a meta-level in order to orchestrate a GenAI process, determine if what comes back has utility, and iterate correctively until the task is complete. Anybody with less than Level 5 is increasingly at a disadvantage, and I would argue that the reduced facility with language makes that disadvantage very difficult to avoid unless you ascribe near-mythic powers to LLMs. If LLMs indeed had such powers, it would beg the question of why low-skill human interactions were of value to task execution in the first place. More likely is people in Level 4 feeling pressure to up their game.

Take a moment to line that up with AI vendor marketing on how it will democratize knowledge and skill. Add in all the people who make online boasts about what LLMs enable for them. Some of it will indeed be true, so for the sake of argument let’s stipulate to all of that potential and be full-throated in our optimism. Imagine a big, warm, fuzzy percentage of humanity that will experience this brave, new, ever-expanding world of possibility… and hold my beer. I’m going to go and fetch the literacy data.

The Data

I’ve set up a Git repository to provide two Excel workbooks that I downloaded via the PIAAC Data Explorer:

• PIAAC-2012-2023-by-sex-and-age.xlsx: The data over three reporting periods, with some break-down by sex or by 10-year bands of age range.

• PIAAC-2023-aggregate.xlsx: Summarized numbers from the most recent reporting period, without any demographic break-down.

The key findings from that second file are:

Year/Study	  PIAAC 2023
	
Jurisdiction	  U.S. Household
	          (16-74 years old)
	
Proficiency	  Percentage
Below Level 1	    11.99
Level 1	            16.74
Level 2	            29.36
Level 3	            29.81
Level 4	            10.88
Level 5	             1.22

There is the reality. That third band combining Levels 4 and 5, representing people who would have some hope of participating meaningfully in an AI-heavy economy: 12 percent. And of that 12 percent, only a little over 1 percent even have the language potential to excel at it, and mere language proficiency alone would never be enough. For the people who have hope, 1 in 12 start with some minimal skill suggesting they might warrant that hope, and the other 11 are scrambling to play catch-up from a position of measurable but possibly repairable deficit. The other 88 don’t even know the race track exists.

Let that rattle around the brain pan for a bit. For those holding the most optimistic view on GenAI, if current literacy patterns continue, then we could face an outcome where 12 percent of the population go in one direction, and 88 percent go in an entirely different one. The only thing that would easily cast the situation in a less dire light would be if something close to 88% of the population currently worked in jobs that don’t have the potential for being cannibalized by AI, such as some blue-collar skilled trades and most unskilled labor.

Unfortunately, according to the AFL-CIO, something like 58% of the US workforce is estimated to currently be white-collar.

It is the basis for a painful transition process into a future combined economic and political divide that will make the current landscape seem a cake-walk by comparison.

This is one of several reasons why I’ll keep saying that we need better options. The search is and will be time-consuming. If you’re finding the material at all compelling, you can vote for more with your dollars on Substack!

The Experimentalist : On the Turn of a Phrase © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

This series will take some time, largely because we are faced with some very non-trivial economic and career challenges. I don’t want to drag out the “so what?” indefinitely so I’ll summarize where we are at and where we are headed. As this is just an act of literary mercy, the points made in this post will lack deep development. Substance will follow later.

Prev: AGI is Irrelevant | Next: On the Turn of a Phrase

So far we’ve covered:

• The economic and career landscape is changing.

• Whatever the substantive story about AI at present, the simple reality is it is—and will continue to be—a growing factor in those changes.

• The easy view on AI is that it will provide valuable performance benefit to those employees using it.

• The subtler view on AI is any such benefit experienced by an employee may be short lived; that very use case identifies the next likely target for an even better AI.

• The impacts may be less about AI itself, and more about how large companies view it as a staff reduction tool.

• Government is unlikely to do anything to protect employees, and even if it did, it would be unlikely to take such action in a timeframe that matters.

Ok, so much for the sounds of feet marching to the drum-beats of doom. We’ve only outlined the problem, not dictated humanity’s future as some unavoidable fate.

The thesis that will be explored is straightforward to articulate, but the details and the supporting arguments will take work. It will be a bit of a “tough love” process, but reality is what it has been shaping up as, not as we would have it be.

1. We’re going to have to “up our game” just to stay in the game at all. That applies to any dimension where AI is weaker, but those dimensions will be a moving target as AI improves.

2. Even if AI wasn’t a factor in the change we are experiencing, the economic position of the workforce has been eroding for a long time. We need to clearly understand that and tackle it directly as a problem in its own right.

3. Social media, traditional media, and the political landscape all provide a nightmare landscape of distraction. It is critical to reduce the impact of that so we have the bandwidth to tackle the first two points. The man behind the curtain would rather we not twig to the distraction dynamic, but we must manage it.

4. If we can pull all of that off then maybe we make a better, stronger human experience out of this. I believe there are past models we can look to for ideas. That will be explored in later articles.

5. Finally, I have my own “up my game” journey that will result in other entire series of articles. For all my data engineering and observability buds, I haven’t forgotten you. I’m hoping that, to some degree, these articles embody elements explored in Hammer & Anvil.

There is your ten-thousand-foot view of where we are in this series now, and what is to come. Thank you for allocating your reading bandwidth and being part of the journey. Now, as every great journey deserves a proper soundtrack, enjoy!

The Experimentalist : The Foreshadowing © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

Over the past year as I increasingly focused on the incoming and inexorable volcanic lava flow that is the LLM slice of AI, much like everybody else it was hard not to be captured by the various dimensions of the discussion and technology. Particularly with a social media and political landscape purpose-engineered to be sources of distraction, sometimes it can take awhile before you realize you need to step back. You must let the details drift away to begin to see what is solidifying, and what is burning to ash.

Prev: And So It Begins | Next: The Foreshadowing

One of the more contentious debates has been around the capacity of LLMs to reason, and what the future holds with Artificial General Intelligence (AGI). Many claim that LLMs already reason, and conversely many claim that the LLM model itself is fundamentally mathematically insufficient as a platform to ever achieve reasoning.

I lean a bit towards the latter, in part due to humanity’s intense wiring for apophenia. We would deify a dog’s left nut if we could convince ourselves we would then feel better about the chaotic world around us. That said, there are some topological data quirks and a host of LLM ecosystem improvements to increasingly allow for. LLMs cannot be underestimated, and their capabilities are a fast-moving target. I also don’t think the debate much matters in terms of societal impact. Neither reasoning nor AGI are what will cause the anvil to hit the hammer for humanity.

It will be the decisions made by those with the power to broadcast those effects to thousands and tens of thousands of people at a time.

The Evolving Impact

Some companies have already become talking points through how they reacted to the potential of LLMs. IBM, for example, laid off eight thousand H/R staff because of a belief in what LLM-based automation would immediately provide. It made IBM short-lived joke fodder because they rapidly had to re-hire many of those employees. Other companies like Duolingo and Klarna had similar missteps. The result was that the anti-LLM crowd laughed and felt that their position was substantiated.

Unfortunately, an import point was being missed. The corporate world had tipped its hand on strategy, and that strategy is what will actually matter going forward.

The layoffs, and similar recent policy statements by many other companies, made it very clear that the difference between employment and unemployment for hundreds, even thousands of people, depended on a finger just itching to pull the trigger. Entire categories of staff that had invested portions of their lives in those firms were unwanted, and past visions of “company culture” were laid waste as those employees learned that their very participation had long been silently begrudged. The only difference that AGI could make in the future, is to measure fate in terms of “how soon” and “how many.” Even if LLMs today are found to not achieve some necessary performance threshold, it will simply motivate companies to spend their way into achieving that goal. Careers in many large companies from now on will, at best, be measured a fiscal quarter at a time.

The Reckoning

We have barely begun to grapple with what this will mean to ourselves individually and the economy broadly. Career progressions or equity-based compensation packages associated with longevity now warrant a radical risk-adjustment. Personal debt and savings management strategies elevate from “good practices to work towards” to “critical for survival on short notice.” The meaning of skill itself becomes a very complicated issue in a world that rushes to homogenize many career paths into their simplest interpretations as low-variance and thus automatable commodities.

Some may view the situation differently, and construe change as opportunity. They aren’t wrong, or at least not wrong for awhile. Unfortunately, some of that opportunity is just the next arbitrage opportunity that corporations will harvest with a slightly better AI model. The value proposition of LLMs is that the manual effort input is less than what the manual effort would be to create the resulting output. That input effort is paid for in staff time, and thus will always be the next obvious target for automation as it would eliminate both the doers and likely their supervisors simultaneously.

Any view held that somehow large corporations or governments will buffer people from these forces, better have some practical empirical examples to point to. It took fifteen years for most governments to begin grappling with cryptocurrency, and AI is moving faster than crypto did. When companies lay off thousands at a time, they are not examining your individual skills or contributions to make that call. They’re just drawing a line and saying “go!” to those on the wrong side of a corporate demographic. Governmental policy mechanisms to restrain that are minimal, and unlikely to change much in the near future.

So far as I can see, at this moment, we’re all on our own…. and it’s time to have a much better plan in place.

The Experimentalist : AGI is Irrelevant © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

The remainder of the twenty-first century is going to be a time unlike any we have seen for generations.

The forces that will shape it have been building for decades. Some are economic and will restructure how we navigate our financial well-being. Others are technological, likely to entirely reshape educational paths and career identities. Threading through both are political dynamics that no longer supply a slow-moving, stable framework of laws and norms to buffer other processes so they evolve with less personal and societal risk.

Next: AGI is Irrelevant

We are all going to find ourselves, sooner or later, between hammer and anvil. Many of you are there already, feeling new patterns being beaten into your lives. Whatever will be forged, one thing that seems increasingly clear is that formulae of past years — our life strategies — may no longer apply. It is up to us to do what we can to guide our re-shaping.

For many years I wanted to start a newsletter. Mostly I imagined it being some accumulation of technical and job-related ramblings. Some of it will remain so, but not all of it. The times have become larger, and more consequential, than I had planned on. I want this to be what I hope is one of many efforts, by many people, to figure out how to make what comes next be a positive thing in the historical record of the human experience. Right now it isn’t a given that this is what we will see, at least not for awhile, so we need to get to work.

Welcome to the start of the discussion. Plan on leaning in.

The Experimentalist : And So It Begins © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0