Technology

Sort of. We know 'how it works' to the extent that it was engineered with a particular method and purpose. The problem is that it's incredibly difficult to gain any insight into what's 'inside' the network once the data has been propagated through it.

Visualizing a neural network can look a little bit like a constellation of stars. Each star is a node and is connected to other nodes. When given an input, each node makes a small calculation and passes the result to the other nodes they are connected to. The calculation is modified by the connection (by what is called a weight), and the results of the calculations change the weights of the connections. That's what's in the black box.

The constellations in an LLM are very large (the first L in LLM). Each 'layer' may have hundreds of nodes, each of which is connected to every node of the next layer. If there are 100 nodes in two adjacent layers, that makes 10,000 connections. There are many layers in an LLM.

Notice that I didn't mention anything about the nodes or the connections storing any data. That's because they don't, at least in the sense that we're used to thinking about it. There doesn't exist a string of text that says 'Bill Burr's SSN is ###-##-####'. It's just the nodes that do the calculations, and the weights of their connections.

So by now you can probably see why it's so tricky to determine what's 'inside' a neural network, because really it's a set of operations instead of a set of data. The most reliable way to see what it does (so far) is to put something in and see what comes out.