Software Ate the World: Neural Databases Will Eat Software

10 Aug 2023

10 Aug 2023 by Luke Puplett - Founder

Silicon Valley once promised software would eat the world. Now software itself faces consumption by an invasive species - artificial intelligence.

The proliferation of code transformed society. But the ascendance of AI now threatens the coders and companies it enriched. New paradigms emerge...

Intelligent algorithms relentlessly digest data, deriving their own logic. Machine learning subsumes what once required human programmers. A new generation of AI databases self-generate interfaces anticipating user needs.

Winners and losers take shape as traditional applications become obsolete. Established workflows face disruption yet again.

Though the greatest disruption may be to the human mind as our tools encroach upon domains once deemed our exclusive dominion...

The early rumblings of the AI revolution portend tectonic shifts ahead. But revolutions create opportunities for those with vision to see beyond the horizon. The future favors the bold.

Join us as we explore the gathering winds of change, and uncover principles to navigate what lies ahead. The choices we make today will resonate for decades hence...

Why We Create Software at Work

Software has become deeply integrated into the fabric of modern life because of its immense power to transform how we work, socialize, and conduct business. But at its core, business software provides several fundamental forms of utility that deliver enormous value:

Automating Workflows and Processes

Much of software's impact stems from its ability to encode complex logical processes and rules that allow tasks to be performed automatically. Rather than requiring human evaluation and decision making for each step, software codifies workflows so they can be handled efficiently at scale. Examples include processing transactions, routing approvals, transferring data between systems, generating reports, and triggering notifications. This automation of repetitive tasks boosts organizational productivity and consistency.

Customizing Systems for Specific Needs

While some software takes a one-size-fits-all approach, custom-built applications allow businesses, groups, and individuals to achieve their unique goals. Tailoring interfaces, business logic, integrations, analytics, and capabilities to match an organization's priorities and established ways of working provides strategic value. Custom systems increase adoption and ROI by aligning tightly to specialized workflows and competitive needs.

Enabling Data Analysis, Insights for Decisions

Software unlocks the value hidden within data. By collecting, processing, organizing, analyzing, and visualizing information, software reveals key trends, patterns, and insights that would otherwise stay invisible. Analytics and business intelligence software empower data-driven decision making. Data mining tools find correlations and make predictions. The data insights unlocked by software boost efficiency, strategy, and innovation.

While software continues to expand into new domains, these core strengths - process automation, customization, and analytical power - explain why organizations continue to invest heavily in developing and refining business-critical applications. Software provides the magic behind the scenes to enhance what humans can achieve.

Collaboration and Communication

This can include tools for instant messaging, video conferencing, file sharing, and project management. Communication and collaboration software can help to improve productivity, reduce errors, and boost morale

Coordination

Coordination and scheduling software helps businesses to schedule and visualize work in progress, and foster shared understanding across different teams and departments. This type of software can be used for a variety of purposes, including project management, ERP, and shipping.

Controllers and Hardware Instrumentation

Hardware controllers and instrumentation software play a crucial role in managing and optimizing a wide range of physical systems and processes. These tools are designed to interact with and regulate hardware components, machinery, and devices in various industries. Their functions extend beyond traditional computer systems and encompass diverse applications.

Marketplaces

Business marketplaces stand as dynamic platforms reshaping commerce paradigms. By seamlessly connecting buyers and sellers, they streamline transactions, offer diverse options, and provide cost-effective solutions. These digital ecosystems not only enhance efficiency but also amplify market reach for sellers while empowering buyers with an array of choices.

Significantly, specific software categories like embedded controllers, Internet of Things (IoT) applications, and device drivers exhibit inherent attributes that render them less vulnerable to AI displacement. These software types necessitate instantaneous responsiveness, intricate hardware integration, and precise low-level management, posing challenges for replication through AI technologies, and, in some cases, rendering AI's endeavor futile. Similarly, business marketplaces, while possessing attributes that resist AI substitution, could face transformation through automated bargaining bots. These bots, armed with advanced algorithms and AI, might navigate negotiations and transactions, potentially reshaping marketplace dynamics. While AI may not fully supplant these intricate software domains or marketplaces, it could undeniably catalyze their evolution.

In another vein, a significant portion of business software operates on shared modes of information storage, retrieval, event orchestration, and scheduling. This foundational similarity, while vital for diverse functions like customer relationship management, supply chain coordination, and financial tracking, also renders these software categories potentially susceptible to AI replacement. As AI algorithms advance, they become increasingly proficient at managing these core operations, raising the possibility of automating tasks that currently rely on human intervention. Consequently, while AI's influence might not fully supplant the intricacies of specialized software domains or marketplace dynamics, its impact on software with these common modes of operation remains a transformative force, reshaping conventional workflows and expanding the boundaries of automation.

How AI Could Displace Traditional Software

While software has become central to business and society, the rise of AI threatens to disrupt traditional software development. As AI grows more sophisticated, "smart" systems powered by machine learning could start replacing manually coded applications:

Intelligent Systems Replace Explicit Programming

Rather than requiring human developers to specify explicit instructions, AI-powered systems rely on recognizing patterns and inferring logic from training data. By analyzing large datasets, deep learning algorithms can derive their own programming suited to a problem domain. This reduces the need for static, hand-crafted software when adaptable machine intelligence can replicate and exceed its capabilities.

The AI Database that Renders its Own Interface

An advanced AI system could ingest raw data and use its analytical skills to build its own interfaces and visualizations. Natural language processing allows conversational queries. Graphical presentation of data relationships and insights further reduces the need for predefined reporting tools. The AI system becomes both database and flexible front-end in one.

This has the potential to revolutionize the way we interact with data.

Currently, most databases require us to use a pre-defined interface to query and visualize data. This can be limiting, especially for users who are not familiar with the database's syntax or who need to perform complex queries.

An AI database that renders its own interface would be able to understand our natural language queries and generate visualizations that are tailored to our specific needs. This would make it much easier for us to explore and understand data, even if we are not experts in data analysis.

In addition, an AI database could be used to build conversational AI applications. In fact, it would already be a universal conversational AI application. This would allow users to interact with data in a natural way, without having to learn any complex syntax, or need a software developer to write such syntax or indeed build a UI around it.

For example, imagine a user who wants to know how many sales were made in the past month. They could simply ask the AI database, "How many sales were made in the past month?" The AI database would then understand the user's question and generate a visualization that shows the sales data for the past month.

What's more, the AI database, as well as having its neural net trained on the data, it could structure and restructure it at will, index it and write code against these structures, ad-hoc, if it feels this would be the best, most accurate and trusted way to answer a question.

The AI database could also be used to build predictive models. This would allow users to predict future trends and make better decisions.

For example, an AI database could be used to predict customer churn. This would allow businesses to identify customers who are at risk of leaving and take steps to prevent them from doing so.

Finally, such the AI database could build narratives and interactive data journeys in three dimensions, perhaps to be explored using VR headsets. From framed in the context of the AI database and how easily it will be able to portray complex things visually, it's easy to see why we need to move away from flat, 2D screens into more immersive experiences. It's also easier to understand how interacting with such systems may move to interacting with avatars; all-knowing superhuman colleagues.

Make no mistake: this transition is upon us and unstoppable.

Winners and Losers in an AI World

As AI subsumes more software functionality, winners will include cloud providers selling access to powerful computing resources and companies wielding unique proprietary datasets. But traditional software firms relying on licensed code could lose ground to open AI platforms. Coders must adapt their skills to train, refine, and oversee intelligent systems. Jobs focused on repetitive manual tasks are at risk. But consumers may benefit enormously from the new capabilities unlocked by AI.

In the intricate realm of software development, an often-underestimated reality emerges: the majority of code resides within the interface. Graphically presenting information, collecting user inputs, guiding their journey, safeguarding against errors, and ensuring proper utilization constitute a significant portion of a codebase's architecture. This graphical facade, carefully crafted to offer seamless user experiences, conceals intricate complexities that seamlessly orchestrate interactions between users and the underlying system.

Equally substantial is the code dedicated to the management of data. Reading and writing data, especially while accommodating the demands of concurrent users, ensuring data integrity, synchronization, and responsiveness requires a robust backend that can handle the complexities of a dynamic user base.

Yet, it's intriguing to observe that the core application logic, the intellectual nucleus guiding the software's functionalities, constitutes merely a fraction of the overall codebase. The code that forms the software's raison d'etre is overwhelmed by all this other necessary but non-core-value code.

The key point is that AI will transform how software delivers value - from hand-crafted logic specified by developers to an emerging machine intelligence trained on data. Rather than battling this trend, forward-looking companies should embrace AI's software-eating potential.

The Interlocking Nature of Workflows, Jobs, and Software

To understand how disruptive technology can make entire software ecosystems obsolete, it's important to recognize the interdependencies between workflows, jobs, and systems that evolve together over time:

Workflows, Jobs, and Systems Tightly Interlock and Co-Evolve

Business processes and software tools rarely exist in isolation. Workflows span departments, with handoffs and integrations between systems. Data and credentials are shared across applications. So systems design and procedures develop organically around these connections. The result is a complex web of dependencies not easily untangled.

Once workflows are set, jobs take shape around them. Roles and responsibilities align to fit the process steps. Skills and training focus on the specific activities needed within the broader workflow. Individual incentives, metrics, habits, and organizational culture adapt accordingly. The system influences the jobs, which then perpetuate the system.

Legacy Systems Entrench Inertia

Over time, processes become ingrained and codified into legacy systems and technical debt accumulates, creating organizational inertia. Outdated workflows persist even when there are clearly better approaches available.

For example, paper-based resumes endure when LinkedIn profiles are more accessible. Fax machines remain common despite digital communications. Handwritten signatures are required when e-signatures could be more secure. Older techniques like check payments stay entrenched compared to electronic funds transfer.

Migrating off legacy platforms requires time, cost, and risk. So businesses cling to antiquated systems long past their expiration date. Only an external disruption - like a shift in the labor market or new technology - breaks the inertia by making the opportunity cost of staying put unbearable.

This coevolution of workflows, jobs, and systems creates inertia. Transforming any component requires rethinking how all the pieces interconnect. Overcoming inertia means diagnosing and reengineering processes holistically, not just swapping old technology for new.

My HTML Resume Hits Hiring Portal Roadblocks

As a web developer, I decided to publish my resume or CV as an HTML webpage to showcase my skills in action. The interactive page highlights my projects with semantic markup and links.

But despite this dynamic web version, I still need to generate a plain PDF copy for most job applications. Recruiters tell me their clients' applicant tracking systems only accept uploaded PDFs, not HTML pages. So my multimedia resume gets reduced back down to lifeless black and white text.

The rationale is often that PDFs provide a standardized, printable document. But in reality, it demonstrates organizational inertia. Even though more accessible and visual resume formats exist, established workflows haven't caught up. Unwillingness to modify old systems creates friction for applicants and recruiting agencies alike.

It reminds me how hard it is to overhaul entrenched workflows, even when technology offers clear improvements. The accumulation of legacy processes and technical debt acts like gravity, maintaining the status quo until a major disruption occurs. My HTML resume will remain ahead of its time until hiring platforms evolve or new players disrupt the space.

We Shape Our Tools, Thereafter They Shape Us

"First we shape our buildings; thereafter they shape us." This Winston Churchill quote rings true for software tools. The systems and workflows we construct take on a life of their own. Legacy processes and technical debt accumulate, leading to inertia.

This inertia creates opportunity for startups unburdened by existing paradigms. But disrupting established players still proves challenging. The hardest task for startups is not designing better products, but driving adoption. First-time founders obsess over product, while repeat founders focus on sales and distribution.

As Marc Andreessen noted, "There are no bad ideas, only bad timing." The same vision may fail in one era yet succeed in the next if conditions align. Timing is crucial.

The lesson is that while ingrained workflows give incumbents strength in exploiting existing markets, they also impose inertia resisting new ideas. Startups possessing the clarity to rethink paradigms and patience to achieve product-market-timing fit can unlock major disruptions.

But leaders should not wait for external disruption. We must proactively reinvent legacy systems before inertia fully takes hold. The buildings, tools and workflows providing an advantage today could drive failure tomorrow unless periodically reimagined. Avoiding inertia requires retaining a beginner's mindset even at the heights of success.

When Entire Chains Become Obsolete

Once a tipping point of disruption occurs, it can set off a chain reaction dismantling once-essential workflows and systems from start to finish:

Travel Agents - The rise of airline/hotel booking websites and mobile apps allowed travelers to compare options, purchase tickets, and make reservations directly. Travel agent roles declined sharply as booking workflow needs evaporated.

Video Rental Stores - Services like Netflix, Hulu, and Prime Video digitally stream entertainment, disrupting the entire process of visiting a video rental store, searching for titles, checking out physical media, and returning items on time.

Stock Traders - Electronic trading platforms allow algorithmic systems and retail investors to execute trades in seconds based on signals and strategies. This reduced need for human traders making deals over phone and pit trading floors.

Typists - Word processors and email cut out the intermediary roles of typists who would transcribe documents and memos as dictated by management. Secretarial pools disappeared as executives handled drafting work directly.

Film Photography - The shift from analog film cameras to digital photography disrupted workflows around purchasing film, using darkrooms for development, and manually processing prints. Digitization made many of those skills and jobs redundant.

In each case, a breakthrough technology so transformed a domain's workflows that whole chains of related skills and systems were suddenly obsolete. Established processes that once felt essential evaporated almost overnight.

This demonstrates how the right disruption can unravel even entrenched ways of working, unlocking greater efficiency and innovation in the process. But as workflows transform, so must the software systems, business processes, training, and metrics designed around old paradigms.

Categorizing the Diversity of Software Applications

The range of software tools created highlights the multitude of problems they aim to solve. Here are some major categories of applications:

• Collaboration/Productivity - Facilitate teamwork and workflows (e.g. Office 365, Slack, Asana)

• Marketplaces - Connect supply and demand, enable transactions (e.g. Amazon, Uber, eBay)

• Analytics - Collect, process, visualize data for insights (e.g. Excel, Tableau, SAS)

• User Experience - Improve interfaces and ease of use (e.g. web and mobile apps)

• Personal Applications - Assist with scheduling, finances, habits (e.g. Mint, Evernote)

• Storage and Sharing - Save, organize and share files and data (e.g. Dropbox, SharePoint)

• Custom Enterprise Software - Address specific organizational needs (e.g. ERP, CRM)

• Industry-Specific Software - Tailored systems for verticals like healthcare, finance, engineering

• Infrastructure - Foundational platforms like operating systems, databases, cloud computing

• Entertainment - Games, video/music streaming, social media (e.g. Spotify, Candy Crush)

This taxonomy highlights the sheer diversity of roles software plays in our lives - from optimizing workflows to enjoying leisure time. Each application category solves a set of related user problems. But fundamentally, they aim to make users more informed, connected, productive, or entertained.

Aligning Software to Core Human Activities and Needs

At its best, software empowers activities that enrich our lives and actualize human potential. Analyzing core human motivations shows how software can fulfill our highest needs:

• Physiological - Tools for procuring food, scheduling rest, monitoring health (e.g. food delivery apps, sleep trackers, telemedicine)

• Safety and Security - Systems for stability, law, identity protection, and governance (e.g. cybersecurity, encryption, digital ID, voting)

• Social - Applications for connection, communication, and community (e.g. social media, video chat, multiplayer games)

• Esteem - Platforms for learning, creativity, and sharing professional achievements (e.g. online education, design tools, LinkedIn)

• Self-Actualization - Technologies that expand capabilities and transcend physical limits (e.g. AI, VR, human augmentation tech, space exploration)

• Self-Expression - Creative tools for crafting, designing, making (e.g. Photoshop, AutoCAD, music production software)

• Meaning - Software for convening around causes, spirituality, ethics (e.g. community organizing platforms, meditation apps, wisdom sharing)

Rather than just boosting economic output, we should judge software by how effectively it nurtures human development, builds connections, and provides fulfillment beyond material consumption. Tools aligned with our highest motivations will uplift humanity to new heights.

The Double-Edged Sword of Disruption

The accelerating pace of technology gives us tools that are like a double-edged sword - able to build or destroy depending on how we wield them:

On the one hand, innovations like AI, networks, and biotech expand human capabilities exponentially. We can cure disease, expand knowledge, reduce poverty, and explore the cosmos at unprecedented speed and scale.

But without wisdom, technology can enable frightening scenarios - climate catastrophe, surveillance states, autonomous weapons, engineered pathogens.

Disruptions making old ways obsolete are inevitable with progress. But transitioning from one equilibrium to the next without existential risk requires actively shaping the trajectory of advancement.

This makes the ethics of technologists paramount. Software should empower human flourishing, not our worst instincts. System designers must consider second-order effects on justice, universal rights, human dignity, and stewardship of the planet.

Progress cannot be stopped, only guided. But wise governance of technology and fair distribution of its gains are crucial to direct disruption toward human thriving rather than doomsday dystopia. Our tools amplify both the best and worst in human nature. The choice is ours.

Embracing the Pace of Change

Rapid technology change can feel unsettling. But history shows societies adapt through generational shifts:

New workflows and tools first get pioneered by the youth, early adopters, and startups unencumbered by legacy systems. As benefits accumulate, methods diffuse more broadly.

Eventually, new habits reach a tipping point where the mainstream abandons ingrained but outdated approaches. This progression recycles as even newer technologies emerge.

Businesses and institutions must strike a balance between stability and nimbleness. They need the wisdom to know which deeply embedded workflows have reached the point of counterproductive inertia.

But preparing people and systems for continuous incremental change makes largescale disruptions easier to absorb. Some guidelines include:

• Architecting modular, adaptable systems

• Maintaining talent mobility across roles

• Incentivizing regular upskilling and reskilling

• Proactively monitoring the competitive landscape

• Running small experiments to detect shifts

Exponential technology promises an intense period of transformation. But by studying disruption patterns and promoting organizational flexibility, we can smoothly transition to each new equilibrium.

The future will keep arriving. Our challenge is to gracefully evolve with it.

Conclusion

The trajectory of technology points toward an era of ambient computing - where software fades into the background and environments come alive with responsive intelligence. This promises to reshape society by enabling ubiquitous access to information and AI assistance.

This subsumption of software into smart systems promises to reshape society on the scale of electrification or the Internet. It will enable ubiquitous access to information, seamless remote collaboration, and AI assistance integrated into everyday processes. Legacy applications will give way to conversational platforms anticipating our needs.

Realizing this ambient potential requires overcoming inertia around entrenched systems and workflows. It demands envisioning paradigm shifts rather than just incremental progress. The companies and countries quickest to embrace this human-centric evolution will lead the coming revolution.

In the prescient words of Mark Weiser, father of ubiquitous computing, "The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it." As software evolves from visible tool to invisible assistant, its highest purpose remains serving human priorities. Our challenge is dreaming big enough to guide technology toward that vision.

With AI dissolving many current software challenges, our brightest minds will be freed to solve humanity's hard problems again. Challenges of the physical world return to the forefront - disease, energy, space, climate, governance.

By overcoming inertia and embracing ambient intelligence, we can hasten the day when technology recedes into the background and the gritty problems of the tangible world recapture our attention. Software eating software frees us to nourish our shared humanity.

---

Learn more