Readings

voidStar

Mar 1, 2023 • 4 min read

Things I read/skim read today...

Summary

Simplicity Wins: How Large Language Models Will Revolutionize Software Engineering

Software engineering is on the brink of a revolution with the emergence of large language models (LLMs). LLMs are AI systems that have been trained on large amounts of data, allowing them to generate natural language text and source code. LLMs allow developers to specify intent using prompts, rather…

BlackHC's AdventuresAndreas Kirsch

Food for thought about how LLMs are going to impact how we develop software.

Infinite AI Array

An interesting look at integrating ChatGPT with programming language data structures.

A good illustration of the power of constraints and limiting scope and the power of having a strong foundation to build upon.

Details

Simplicity Wins: How Large Language Models Will Revolutionize Software Engineering

BlackHC's AdventuresAndreas Kirsch

The emergence of LLMs in software engineering can be seen as a manifestation of Richard Sutton’s “bitter lesson” in reinforcement learning research, which states that simpler approaches that scale well with more compute will eventually outperform more complex approaches. In the context of software engineering, LLMs offer a potentially simpler and more scalable approach to implementing complex tasks, such as writing and debugging code.

Language models are changing the way we write software, allowing us to focus more on intent and less on implementation. By adapting to these changes and leveraging the power of LLMs, we can avoid the “bitter lesson” of being left behind as simpler approaches outperform more complex ones.

Infinite AI Array

The Problem Statement
All great tools solve a problem.

Lists end

Dictionaries don’t have every key

I don’t have enough ideas

List operations usually happen right away and sometimes I’d like them to take a second or two so I can appreciate the effort involved in programming

My types and values are consistent like a program, not expressive and eclectic like natural language

A good illustration of the power of constraints and limiting scope:

Removing the online multiplayer is a massive reduction in complexity. Shelving the loosely defined in-game economy makes the game a lot simpler. And if we surrender the 3D constraint, I’m pretty confident we can make a 2D Zelda-like game that he and his friends can play in a browser or we could load it onto a cartridge that plays in an actual Gameboy.

...

Going full retro handheld has other advantages: it allows my son to help more in the process. He’s a pretty good pixel artist and MIDI composer, which GB Studio has an awesome (and familiar to him) piano roll composer tool. I should be able to say “draw this” or ask him to riff on some biome/dungeon music tracks. It’d be great if he could put his thumbprint on it.

...

The hardware constraints of the Gameboy also provide some incredible guardrails against scope creep. 4096 KB max cartridge size constraints, map size constraints, tritone color constraints, number of NPC “actor” constraints, and variable constraints narrow your focus down to what you want the game to do. Each scene of GB Studio has a counter for how many objects you have left. Your graphics can only be three colors (black, white, dark green) and I saw somewhere you only get 132 variables, which is a wonderful forcing function. If a feature is getting too complex, I can say, “We literally don’t have enough variables to make that idea.”

Also touched upon is the power of having a strong foundation in place:

I think with these main pieces blocked out, we can —as Shigeru Miyamoto says— “get the fundamentals solid first, then do as much with that core concept as our time and ambition will allow.”

The blog post also generously links to the resources used to make the game, including links to documentation, tutorials and art assets.

Summary

Details

Sign up for more like this.