chris<p><strong>How I use LLMs – meat tricks with Simon’s LLM command</strong></p><p><em>Earlier this year I <a href="https://www.thegreenwebfoundation.org/publications/report-ai-environmental-impact/" rel="nofollow noopener" target="_blank">co-authored a report about the direct environmental impact of AI</a>, which might give the impression I’m massively anti-AI, because it talks about the signficant social and environmental of using it.</em> <em>I’m not. I’m (slowly) working through the content of the Climate Change AI summer school,</em> <em>and I use it a fair amount in my job. This post shows some examples I use </em></p><p>I’ve got into the habit of running an LLM locally on my machine in the background, having it sit there so I can pipe text or quick local queries into it.</p><p>I’m using Ollama, and the Simon Willison’s wonderful <code>llm</code> tool. I use it like this:</p><pre><code>llm "My query goes here"</code></pre><p>I’m able to continue discussions using the <code>-c</code> flag like so:</p><pre><code>llm -c "continue discussion in a existing conversation"</code></pre><p>It’s very handy, and because it’s on the command line, I can pipe text into and out of it.</p><p><strong>Doing this with multi line queries</strong></p><p>Of course, you don’t want to write every query on the command line.</p><p>If I have a more complicated query, I now do this:</p><pre><code>cat my-longer-query.txt | llm</code></pre><p>Or do this, if I want the llm to respond a specific way I can send a <em>system</em> prompt to like so:</p><pre><code>cat my-longer-query.txt | llm -s "Reply angrily in ALL CAPS"</code></pre><p>Because llm can use multiple models, if I find that the default local (currently llama 3.2) is giving me poor results, I can sub in a different model.</p><p>So, let’s say I have my query, and I’m not happy with the response from the local llama 3.2 model. I could then pipe the same output into Claude instead (I’d need an API key and the rest set up, but that’s an exercise left to the reader, as <a href="https://llm.datasette.io/en/stable/plugins/directory.html" rel="nofollow noopener" target="_blank">the LLM docs are fantastic</a>)</p><pre><code>cat my-longer-query.txt | llm -m claude-3.5-sonnet</code></pre><p><strong>Getting the last conversation</strong></p><p>Sometimes you want to fetch the last thing you asked an llm, and the response.</p><pre><code>llm logs -r</code></pre><p>Or maybe the entire conversation:</p><pre><code>llm logs -c</code></pre><p>In both cases I usually either pipe it into my editor, which has handy markdown preview:</p><pre><code>llm logs -c | code -</code></pre><p>Or if I want to make the conversation visible to others, the github <code>gh</code> command has a handy way to create a gist in a single CLI invocation.</p><pre><code>llm logs -c | gh gist create --filename chat-log.md -</code></pre><p>This will return an URL for a publicly accessible secret gist, that I can share with others.</p><p><strong>Addendum – putting a handy wrapper around these commands</strong></p><p>I have a very simple shell function,<code>ve</code> that opens a temporary file, for me to jot stuff into, and upon save, echoes the content to STDOUT, using <code>cat</code>.</p><p>I use the fish shell</p><p>This then lets me write queries in editor, which I usually have open, without needing to worry about cleaning up the file I was writing in. Because LLM stores every request and response in a local sqlite database, I’m not worried about needing to keep these files around.</p><pre><code>function ve --description "Open temp file in VSCode and output contents when closed" # Create a temporary file set tempfile (mktemp) # Open VSCode and wait for it to close code --wait $tempfile # If the file has content, output it and then remove the file if test -s $tempfile cat $tempfile rm $tempfile else rm $tempfile return 1 endend</code></pre><p>This lets me do this now for queries:</p><pre><code>ve | llm</code></pre><p><strong>One liner queries</strong></p><p>I’ve also since set up another shortcut like this for quick questions I’d like to see the output from, like so:</p><pre><code>function ask-llm --description "Pipe a question into llm and display the output in VS Code" set -l question $argv llm $question | code -end</code></pre><p>This lets me do this now:</p><pre><code>ask-llm "My question that I'd like to ask"</code></pre><p><strong>Do you use this all the time?</strong></p><p>Not really. I started using <a href="https://www.perplexity.ai/" rel="nofollow noopener" target="_blank">Perplexity</a> last year, as my way in to experimenting with Gen AI after hearing friends explain it was a significant improvement on using Search as it enshittifies. I also sometimes use <a href="https://claude.ai/" rel="nofollow noopener" target="_blank">Claude</a> because <a href="https://support.anthropic.com/en/articles/9487310-what-are-artifacts-and-how-do-i-use-them" rel="nofollow noopener" target="_blank">Artefacts</a> are <strong><em>such</em></strong> a neat feature.</p><p>I also experimented with <a href="https://huggingface.co/chat/" rel="nofollow noopener" target="_blank">Hugging Face’s Hugging Chat</a> thing, but over time, I’ve got more comfortable using <code>llm</code>.</p><p>If I wanted a richer interface than what I use now, I’d probably spend some time using <a href="https://openwebui.com/" rel="nofollow noopener" target="_blank">Open Web UI</a>. If was to strategically invest in building a more diverse ecosystem for Gen AI, it’s where I would spend some time. <strong>Mozilla, this is where you should be investing time and money if you insist on jamming AI into things.</strong></p><p>In my dream world, almost every Gen AI query I make is piped through <code>llm</code>, because that means all the conversations are stored in a local sqlite database that I can do what I like with.</p><p>In fact, I’d probably pay an annual fee (preferably to Simon!) to have my llm sqlite database backed up somewhere safe, or accessible from multiple computers, because as I use <code>llm</code> more, it becomes more valuable to me, and the consequences of losing it, or corrupting it in some way become greater.</p><p>If you have had success using llm that way, I’d love to hear from you.</p><p><a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://rtl.chrisadams.me.uk/tag/ai/" target="_blank">#AI</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://rtl.chrisadams.me.uk/tag/datasette/" target="_blank">#datasette</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://rtl.chrisadams.me.uk/tag/generative-ai/" target="_blank">#generativeAi</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://rtl.chrisadams.me.uk/tag/llm/" target="_blank">#llm</a></p>