Getting Real on AI

I’ve been spending a lot of time lately meeting with people who help lead AI development for their companies. I’ve noticed something jarring about these conversations, a certain tone and energy that just seems… off. It took me a while to put my finger on what it was: No one wants to admit they don’t know.

I can’t remember any technology or technical area being so hard to have a real, grounded discussion about with my peers. And I’m talking about thoughtful people who are normally honest and measured in their communication.

What is going on here? Why is it so hard to have real conversations about where we are in AI? And what does it mean for our space?

Pressure around AI is intense

I think it comes down to one big factor: Pressure. Pressure to demonstrate expertise in a new, high-stakes game.

Overnight, AI has become the most critical and valuable frontier in the technology landscape. It has the potential to reshape entire industries and to make or break companies. Everyone has taken notice: investors, shareholders, company management. In the race to harness the potential of AI, technology leaders find ourselves under great scrutiny. There is an inherent pressure to appear knowledgeable, competent, and ahead of the curve.

Everyone is aware of the dynamics of disruptive innovation, so everyone senses that there’s some sort of advantage to be gained, some sort of race to market leadership underway. And everyone senses that 10 years from now, there will be winners and losers, and case studies on what different players did to succeed or miss out. Already, companies are creating leadership positions with titles like “Head of AI” or “Chief AI Officer” even though no one, including the people filling those roles, really knows what they are supposed to be doing.

And of course, group dynamics come into play. If everyone you know is acting with seeming confidence and certainty, it makes it much harder to be vulnerable and real.

When succeeding in generative AI is framed as a make-or-break challenge with profound implications for companies and careers, it creates immense pressure and fear. Against this backdrop, who wants to admit to not knowing something?

AI is new territory for almost all of us

Of course, the reality is that AI is actually so new to almost all of us! Relatively few people have deep roots in this space. Even fewer have extensive experience developing with natural language LLMs, the brand new wave that has emerged just recently.

The very nature of AI development is a factor here. It is simply so unlike more traditional software development, where you typically have a clear design objective and must painstakingly develop the solution from beginning to end. When working with AI models, so much of the challenge is hit-or-miss experimentation. Which prompts work? Which fall short? If you’ve trained your whole career in building software a certain way, this more freeform, experimental model takes some getting used to. It can feel too loose, too improvisational.

Once I’m able to get the people I’m meeting with to drop their guard, they’ll admit their doubts. I’ve had multiple people admit to me that their company’s AI development is “happening in spreadsheets,” meaning that instead of following best practices for building and assessing code, and moving it forward through stages, they are simply trying things with minimal planning or documentation. Others have said that they feel like they are just throwing things at the AI model to see what works.

I’m not surprised that many developers feel the need to re-characterize their AI work in terms that make it sound less like hit-or-miss experimentation, and more like a rigorous, carefully crafted performance. They don’t want to admit to others, or even to themselves, that their process is so unstructured and fluid.

We need more collaboration and honesty

The irony is that we need to help each other more than ever. With AI, we’re in this rare territory where really basic development tools and practices are being developed. Right now, we’re all working on these things independently, which is a massive waste of energy. We should be sharing and collaborating in these areas!

I’m not talking about sharing things that are true competitive advantages or sensitive in any way. I’m talking about us having the honesty, vulnerability, and insight to understand that we can advance the space faster by collaborating around some obvious basics.

And at the very least, we can stop pretending we have all the answers. In any field, when you are more concerned with not looking foolish than with generating something that works, it is very hard to create good work. This is probably even more true in the world of software engineering. When technology leaders feel compelled to act as if they know everything, they may inadvertently close themselves off to alternative perspectives and fresh approaches.

True innovation thrives on the ability to challenge assumptions, embrace failure, and learn from experimentation. So let’s get real… with ourselves, and with each other!