LM summoning

[lmmx]

Thinking about LMs as “summoning” models, the ‘prompt engineering’ aspect can be seen as saying “what’s the word for ___?” (the logic/reasoning research is about making the summoning process justifiable)

If quill is a driver (generating the site as a fixed thing) then a LM could fit into it to provide a “search” mechanism (which is another way of saying ‘summon’ but implying PageRank, a generative model would imply a different verb, and in particular a less deterministic one)

Traditional site indexes take the format of the ptx permuted index (concordance), as given in the traditional index section of a book’s backmatter.

It’s worth thinking about what it implies to use a LM instead of PageRank (eigenvector centrality) and how this embedding differs, thus how it changes space (already starting to replace topic modelling approaches)

In particular, backlinking is central to PageRank (Google was originally called BackRub!), but in LMs the links are intrinsic (you do not need to add hyperlinks between pages, rather the structure is emergent from the text itself)

Doing so accurately is premised on accurate resolution of polysemy (which backlinks avoid the need to by explicitly disambiguated linking) but assuming this is possible there’d be the possibility to ‘summon’ a page on spin.systems without direct naming — cool idea and excuse to experiment with GPT-2 or UCL DARK type logic/reasoning NN methods.

This would take the form of a ‘summoning page’ rather than a ‘search page’, and the output could take the form of either a list of search results or a bifurcating tree (maybe a RF interface to the LM embedding I don’t know?)