{"id":373,"date":"2024-01-16T19:15:58","date_gmt":"2024-01-16T19:15:58","guid":{"rendered":"https:\/\/liverickson.com\/blog\/?p=373"},"modified":"2024-01-16T19:16:34","modified_gmt":"2024-01-16T19:16:34","slug":"extending-ubuntu-via-gnome-shell-extensions-in-2024","status":"publish","type":"post","link":"https:\/\/liverickson.com\/blog\/?p=373","title":{"rendered":"Extending Ubuntu via GNOME Shell Extensions in 2024"},"content":{"rendered":"\n<p>Happy 2024, everyone! December ended up being a bit of a blur for me, because I unexpectedly launched <a href=\"https:\/\/memorycache.ai\/\">MemoryCache<\/a>, made it to the top of the orange website, and have since been exploring new ways to evolve the project ideas that I have related to embedding personal machine learning projects directly into my operating system.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\ud83d\udcda Read more: Why I&#8217;m auto-saving browser documents to augment local AI models and building MemoryCache<\/p>\n<cite><a href=\"https:\/\/liverickson.com\/blog\/?p=330\">https:\/\/liverickson.com\/blog\/?p=330<\/a><\/cite><\/blockquote>\n\n\n\n<p>Before I get into the technical bits that I want to document, the start of a new year feels like a good time to recap <a href=\"https:\/\/iamagoodbing.ai\/running-dalai-small-language-model-locally\/\">my philosophy<\/a> on artificial intelligence: <\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li> Proceed with caution<\/li>\n\n\n\n<li><a href=\"https:\/\/liverickson.com\/blog\/?p=235\">Local<\/a> is <s>better<\/s> the future<\/li>\n<\/ol>\n\n\n\n<p>Since May, I&#8217;ve been reflecting on the metaphors of <a href=\"https:\/\/liverickson.com\/blog\/?p=218\">user agents<\/a> and <a href=\"https:\/\/liverickson.com\/blog\/?p=222\">operating systems<\/a>, and the places that machine learning capabilities can be fit into these existing types of software. While I&#8217;ve been working in the open source community for almost the entirety of my career, the scope of my roles previously were largely at the application and platform layers. In 2023, I set a goal for myself to broaden that in service of increasing leverage that I have as a technologist. Three of my major 2023 accomplishments were: <\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Learning how to <a href=\"https:\/\/liverickson.com\/blog\/?p=236\">build the Firefox web browser from source<\/a>, modify an API, and create an extension to build <a href=\"https:\/\/github.com\/Mozilla-Ocho\/Memory-Cache\">MemoryCache<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/github.com\/misslivirose\/promptly\">Upskill in python<\/a> to fork, run, and tinker with an on-device RAG agent (privateGPT) on my local machine<\/li>\n\n\n\n<li>Switching from Windows to run Ubuntu as my primary operating system, and learn how to write gnome-shell extensions to customize my local compute environment <\/li>\n<\/ol>\n\n\n\n<p>I grew up in the age where the personal computer (&#8220;PC&#8221;) was relegated to Windows computers thanks to catchy commercials from Apple. In practice, the &#8220;personal computing&#8221; market has dramatically shifted from the original visions evolving out of Xerox PARC and the innovation labs of the past into a multi-level mediated&#8230; mess. Thankfully, open source operating systems are thriving, and the barrier of entry into an actually personal computing experience that is tailored to your own environment is lower than ever. <\/p>\n\n\n\n<p>With that in mind, in December, I started looking at how I could start to extend my Ubuntu desktop. <\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Building a GNOME-shell extension <\/h2>\n\n\n\n<p>In the past, I&#8217;ve tried a few different methods of managing some simple tasks and shortcuts to personalize my desktop, including AutoKey and Firefox extensions, but I was feeling an itch to cut out the additional layers of abstraction and go closer to the source. Eventually, I settled on figuring out how to write extensions for the GNOME Shell, which provides the windowing for some Linux systems, including Ubuntu.<\/p>\n\n\n\n<p>GNOME shell extensions can add menus to the status area of the Ubuntu operating system. Often, applications will use this area to add quick-launch items. This menu bar, at least for how it&#8217;s set up on my machine, is always visible, which means that it was a great place to add two-click shortcuts for tasks that I do frequently.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Personal IOT with the Elgato Keylight<\/h2>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"alignleft size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"396\" height=\"288\" src=\"https:\/\/liverickson.com\/blog\/wp-content\/uploads\/2024\/01\/image.png\" alt=\"\" class=\"wp-image-374\" srcset=\"https:\/\/liverickson.com\/blog\/wp-content\/uploads\/2024\/01\/image.png 396w, https:\/\/liverickson.com\/blog\/wp-content\/uploads\/2024\/01\/image-300x218.png 300w\" sizes=\"auto, (max-width: 396px) 100vw, 396px\" \/><\/figure>\n<\/div>\n\n\n<p>While my ultimate goal is to build an agent interface against a privateGPT instance on my machine, I started off with a utility that would control my Elgato light. This felt like an easier, smaller-scoped project to learn the basics of an extension, and it would add a significant amount of utility to me, given that I rarely used my key light before building this interface. You can <a href=\"https:\/\/github.com\/misslivirose\/elgato-gnome-ext\">check out the project and source code here<\/a>.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>The anatomy of a GNOME shell extension can be broken down into: <\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>extension.js &#8211; where the core behaviors of the extension lives <\/li>\n\n\n\n<li>metadata.json &#8211; where the descriptive elements of the extension live<\/li>\n\n\n\n<li>schemas &#8211; a directory to hold the human-writeable and compiled XML that act as storage for the extension\n<ul class=\"wp-block-list\">\n<li>schema_file.xml &#8211; define the variables that can be stored and read from <\/li>\n\n\n\n<li>gschemas.compiled &#8211; the compiled schema file<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>For the Elgato Keylight extension, extension.js does several tasks: <\/p>\n\n\n\n<p><strong>Adds a button to the GNOME taskbar that opens a sub-menu of tasks that can be done<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">      \/\/ Set up the indicator button\n      indicatorButtonText = new St.Label({\n        text: 'Elgato',\n        y_align: Clutter.ActorAlign.CENTER,\n      });\n\n      this.add_child(indicatorButtonText);\n<\/pre>\n\n\n\n<p><strong>Loads in the settings from the compiled schema to restore the light from its last known state<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">let settings, temperature, brightness;\n\nfunction getLightSettingsFromSchema() {\n  let GioSchemaSource = Gio.SettingsSchemaSource;\n  let schemaSource = GioSchemaSource.new_from_directory(\n    Me.dir.get_child('schemas').get_path(),\n    GioSchemaSource.get_default(),\n    false\n  );\n  let schemaObj = schemaSource.lookup(SCHEMA_STRING, true);\n  if (!schemaObj) {\n    log('Error finding saved settings schema');\n  }\n  return new Gio.Settings({ settings_schema: schemaObj });\n}\n\nsettings = getLightSettingsFromSchema();\n\nbrightness = settings.get_int('bright');\ntemperature = settings.get_int('temperature');<\/pre>\n\n\n\n<p><strong>Set up the sub-menu with a button to turn the light on and two sliders to adjust the temperature and brightness of the light<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">toggleButton = new PopupMenu.PopupMenuItem('Turn light:');\n      toggleLabel = new St.Label({ text: lightOn ? 'off' : 'on' });\n      toggleButton.add_child(toggleLabel);\n      this.menu.addMenuItem(toggleButton);\n      toggleButton.connect('activate', () =&gt; {\n        log ('Clicked brightness item');\n        toggleLight();\n      });\n\n      let brightnessPanel = new PopupMenu.PopupBaseMenuItem({activate: false});\n      brightnessPanel.add_child(new St.Label({text: 'Brightness: '}));\n      let brightnessSlider = new Slider.Slider(brightness\/100);\n      brightnessPanel.add_child(brightnessSlider);\n      this.menu.addMenuItem(brightnessPanel);\n      brightnessSlider.connect('drag-end', () =&gt; {\n        updateBrightness(brightnessSlider.value * 100);\n      })\n\n      let temperaturePanel = new PopupMenu.PopupBaseMenuItem({activate: false});\n      temperaturePanel.add_child(new St.Label({text: 'Temperature: '}));\n      let temperatureSlider = new Slider.Slider((temperature - 2900)\/4100);\n      temperaturePanel.add_child(temperatureSlider);\n      this.menu.addMenuItem(temperaturePanel);\n      temperatureSlider.connect('drag-end', () =&gt; {\n        updateTemperature(temperatureSlider.value);\n      });\n\n      this.menu.connect('open-state-changed', (menu, open) =&gt; {\n        if(open) {\n          log ('opened menu');\n        } else {\n          log ('closed menu');\n        }\n      });<\/pre>\n\n\n\n<p><strong>When the items in the sub-menu are changed,  functions call the corresponding command line tasks to update the key light<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/\/ Toggle the light on or off\nfunction toggleLight() {\n\n  log('Calling toggleLight...');\n  if (lightOn) {\n    lightOn = false;\n    GLib.spawn_command_line_sync('keylight-control --bright 0');\n    toggleLabel.set_text(\"on\");\n  } else {\n    lightOn = true;\n    GLib.spawn_command_line_sync('keylight-control --bright ' + settings.get_int('bright'));\n    toggleLabel.set_text(\"off\");\n  }\n}\n\n\/\/ Update the brightness of the light \nfunction updateBrightness(level) {\n  if (!lightOn) {\n    lightOn = true;\n    toggleLabel.set_text(\"off\");\n  }\n  brightness = level.toFixed(0);\n  GLib.spawn_command_line_sync('keylight-control  --bright ' + brightness);\n  settings.set_int('bright', brightness);\n}\n\n\/\/ Update the temperature of the light \nfunction updateTemperature(level) {\n  if (!lightOn) {\n    lightOn = true;\n    toggleLabel.set_text(\"off\");\n  }\n\n  let levelToKelvin = 2900 + level*4100;\n  temperature = levelToKelvin.toFixed(0);\n  GLib.spawn_command_line_sync('keylight-control  --temp ' + temperature);\n  settings.set_int('temperature', temperature);\n}<\/pre>\n\n\n\n<p>This program is a wrapper around a CLI tool that I had already installed (<code>keylight-control<\/code>) but it could easily be updated to interface with the light directly via the Elgato API, something I plan to do in the next few weeks to get more familiar with making API requests through extensions. <\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Setting up a Custom Browser Environment with privateGPT<\/h2>\n\n\n\n<p>My second exploration into GNOME extension authoring was to streamline the process of getting my <a href=\"https:\/\/liverickson.com\/blog\/?p=246\" data-type=\"post\" data-id=\"246\">personal build of Firefox up and running<\/a> without having to use the command line. <\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\ud83d\udcda Read more: Why build a custom version of  Firefox? It lets you test  out new functionality before it makes its way to the actual browser, and gives you more control over how you want your browsing experience  to be.<\/p>\n<cite><a href=\"https:\/\/liverickson.com\/blog\/?p=246 \">https:\/\/liverickson.com\/blog\/?p=246 <\/a><\/cite><\/blockquote>\n\n\n\n<p>Before creating my new extension, I would have to manually run the commands: <\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">cd mozilla-central\n.\/mach run \ncd privateGPT\n.\/run_ingest.sh<\/pre>\n\n\n\n<p>Sure, not a huge number of tasks, but it was enough of a pain that I would find myself deeply unwilling to restart my computer or browser. To be fair, I also had been dragging my feet on signing my custom extension, so there was an additional setup step of having to manually load in the extension through <code>about:debugging<\/code> every time I re-launched the browser.  I&#8217;ve since remedied that. <\/p>\n\n\n\n<p>This extension was even simpler than the Elgato Keylight extension, because it just: <\/p>\n\n\n\n<p><strong>Adds a button to the taskbar<\/strong> <\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/\/ Set up the indicator button\nindicatorButtonText = new St.Label({\n  text: 'Firefox',\n  y_align: Clutter.ActorAlign.CENTER,\n});\n\nthis.add_child(indicatorButtonText);<\/pre>\n\n\n\n<p><strong>Adds a single &#8220;Launch Browser&#8221; button to the menu that drops down<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/\/ Add button to launch application\nlaunchButton = new PopupMenu.PopupMenuItem('Launch Browser');\nthis.menu.addMenuItem(launchButton);\nlaunchButton.connect('activate', () =&gt; {\n  launchCustomFirefoxBuild();\n  initializeDownloadIngestionListener();\n});<\/pre>\n\n\n\n<p><strong>Launches the custom build from the mozilla-central directory<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">function launchCustomFirefoxBuild() {\n    Util.spawn(['.\/mozilla-unified\/mach', 'run']);\n}\n<\/pre>\n\n\n\n<p><strong>Launches the run_ingest script  from the privateGPT directory<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">function initializeDownloadIngestionListener() {\n    Util.spawn(['.\/GitHub\/privateGPT\/run_ingest.sh']);\n}<\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Lessons Learned &amp; Looking Ahead<\/h2>\n\n\n\n<p>This was a really fun project to take on, and I&#8217;m itching to start doing more of this as I think through what kind of ways that I want to expand and customize my personal computing environment. One thing that I&#8217;m considering is figuring out how to make a UI for MemoryCache that lives in this part of my desktop, so that I can do things like:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Quickly verify that items I&#8217;m saving from my browser are showing up in the document store<\/li>\n\n\n\n<li>Get access to new insights that are materializing from the documents<\/li>\n\n\n\n<li>Implement a fast-recall feature to help me find a document that I&#8217;ve recently read<\/li>\n<\/ul>\n\n\n\n<p>One of my biggest takeaways from this was that it was both easier to implement a GNOME shell extension than I thought it was, and also that it required me to practice a lot more patience than I usually have. The first extension took me a few days to build, and just getting the shell of the code to work and get the  item added to the taskbar was a challenge. The <a href=\"https:\/\/gjs.guide\/\">GNOME shell APIs<\/a> have changed from version 42 (the version that I&#8217;m using) and 45 (the latest version of <code>gjs<\/code>), so finding something that worked with my version of Ubuntu took me a while. Once  I had that in place, though, it was much easier to understand how the system worked to draw the menu button, the items in the menu itself, and how to hook up event listeners to the different UI elements. <\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\ud83d\udcda Read more: One of the most helpful resources for putting together a basic GNOME shell extension that I found was a tutorial on Code Project, by user JustPerfection. Thanks JustPerfection!<\/p>\n<cite><a href=\"https:\/\/www.codeproject.com\/Articles\/5271677\/How-to-Create-A-GNOME-Extension \">https:\/\/www.codeproject.com\/Articles\/5271677\/How-to-Create-A-GNOME-Extension <\/a><\/cite><\/blockquote>\n\n\n\n<p>The second takeaway from this project was that it really helped to have someone holding me accountable in a gentle way &#8211; in this particular instance, they didn&#8217;t even realize that they were influencing my motivation to work on the project more! When I had the first piece of the Elgato extension done, it was just a single toggle feature, but I was excited and shared it to Mozilla Social and someone asked for a screenshot. Just being asked to share a screenshot motivated me to learn how to add the whole menu and sliders, and that light accountability was delightful. <\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p>Overall, I&#8217;m very excited to have this as another way of experimenting with computing. I am quite delighted with the idea that we&#8217;re pushing the frontier of casual software development out further and further, and look forward to continuing to explore this in the future.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The &#8220;personal computing&#8221; market has dramatically shifted from the original visions evolving out of Xerox PARC and the innovation labs of the past into a multi-level mediated&#8230; mess. <\/p>\n<p>Thankfully, open source operating systems are thriving, and the barrier of entry into an actually personal computing experience that is tailored to your own environment is lower than ever. With that in mind, in December, I started looking at how I could start to extend my Ubuntu desktop.<\/p>\n<p>This was a really fun project to take on, and I&#8217;m itching to start doing more of this as I think through what kind of ways that I want to expand and customize my personal computing environment. <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":0,"activitypub_interaction_policy_quote":"","activitypub_status":"","footnotes":""},"categories":[3],"tags":[],"class_list":["post-373","post","type-post","status-publish","format-standard","hentry","category-development"],"_links":{"self":[{"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=373"}],"version-history":[{"count":4,"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/373\/revisions"}],"predecessor-version":[{"id":378,"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/373\/revisions\/378"}],"wp:attachment":[{"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/liverickson.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}