GNOME Logs Test Suite

Hello Everyone,

During past weeks, I have been into researching about unit testing as I am now currently working on my third task for GSoC , which is, writing a test suite for GNOME Logs. I would like to give you a brief background about the previous work already done on testing for Logs.

The previous work done is mostly based on testing the Logs frontend using dogtail and behave automated python testing frameworks. You can see more about it here. In my task, I will mostly be working on testing the existing backend functionality in Logs.

I will now try to explain you a little about the existing backend modules in Logs. The Logs backend consists of two modules currently:

• GlJournal: This is the low level module that is responsible for interfacing with the system journal using sd-journal API. It does the work of querying for journal entries and traversing the system journal in the correct order.
• GlJournalModel: This module is responsible for filtering and storing the journal entries according to some specified criteria. These stored journal entries are directly reflected in the frontend modules as well. Moreover, it also handles various attributes to be used in the frontend modules.

I have thought of some initial test scenarios for these modules. I would like to share them here:

GlJournalModel:

• Whether setting the “sort-order” key gives the journal entries in correct order.
• Whether searching works correctly for each of the individual journal fields.
• Whether tokenized mode of search works properly.
• Whether the compressed entries are being counted properly.
• Whether the compressed header is being inserted at the correct position.
• Whether similar entries are being grouped correctly when using event compression.

GlJournal:

• Whether the latest boot is returned properly or not.
• Whether the latest 5 boots are returned properly.
• If the data for journal entries is queried correctly.
• If invalid journal entries are filtered properly (skipping journal entries that are missing necessary journal fields).
• Whether setting matches on the journal works as intended.
• Whether the system journal is traversed in a proper order.

In the coming weeks, I will be implementing these test cases using Glib testing framework. Further, I am exploring the possibility of creating a dummy journal so that predefined journal entries can be added to it for testing purposes.

Apart from test suite, I am happy to tell you that the patch regarding improvements for event compression UI has been merged. Here are some screenshots:

As you can see above, the border colour has been changed so that there is a clear demarcation between the compressed header row and the compressed group. The selected entry is now highlighted with a blue background as shown below:

I would like to thank Jakub Steiner and Allan Day of GNOME Design Team for helping me in further improving the event compression UI and my mentors Jonathan Kang and David King for reviewing and merging the relevant patches.

Moreover, the patches regarding Logs shell search provider have received some further polish. You can follow the progress on it here.

That is all for now. See you in my next blog post 🙂

GNOME Logs: July GSoC progress report

Hello everyone,

In this blog post, I will be talking about various UI improvements to the event compression enhancement and further progress done on GNOME Logs shell search provider during past two weeks. Additionally, I am very happy to say that patches regarding event compression enhancement have been finally committed to the GNOME Logs git repository.

Allan Day of GNOME Design team has suggested some significant UI changes to the event compression feature. You can see them here. I will try to explain them one by one through the use of screenshots.

In the previous design, when the compressed event rows were expanded by clicking the compressed row header, it used to look something like below:

Here, the expanded compressed events were quite difficult to be distinguished from the other uncompressed events. Hence, a colour theme with a dark background for the expanded compressed events was definitely needed. After collaborating with Allan, I have come up with this new darker colour theme for the expanded compressed events:

As you can see, the background colour for both the expanded compressed events and the compressed header row is a bit darker than the previous design. The compressed events counter has a lighter background so that it becomes easy to see how many compressed entries are being shown currently. One thing I like about this design is that it enables the user to focus on the expanded compressed entries instead of the compressed row header, as the compressed header row is eventually a duplication of one of the row from the compressed events.

Another improvement to the expanded compressed events user interface is that previously they were lacking the hover functionality which made them look like they are unclickable. You can see the newly added hover functionality to the expanded compressed events below:

The detailed popover, which shows more information about an event, has also been improved. The new popover uses regular text everywhere instead of a mix of regular text and monospace. The column spacing has been increased and the row spacing is decreased as compared to the previous design. The new detailed popover looks like below:

Another significant UI change regarding event compression is the removal of tooltip showing the event message when hovering above a event row. This has been done as now we use a popover to show the detailed information including the event message.

A working video of the updated event compression feature can be seen here.

I will now be discussing about the progress done on GNOME Logs shell search provider, which is my second task for this year’s GSoC. I have been working on improving over my last year’s patches of the shell search provider. Things have changed quite a bit now in the current implementation of shell search provider as compared to previous year’s.

To start with, for each search invocation, the maximum journal events that are fetched as results during initial search is 5. Fetching such a low number of results shows a great improvement in performance, especially for rotating storage. In the GNOME Logs shell search provider results, the event message is shown as the result name and the process which generated the event is shown as the description of the result:

Another important change to the shell search provider is that now we show results only from the current boot. When a search result is activated, it’s detailed information is shown by a popover pointing to the journal event in the events list. An example of such a window showing detailed information about the activated result is shown below:

As you can see above, we show only the activated result in the events list and a popover containing detailed event information which points to the activated result. We also hide the category list in this new window.

You can get a clear picture of the entire working of the updated GNOME Logs shell search provider in this video.

Next week onwards, I will start working on my third task for GSoC which is developing a test suite for GNOME Logs. A test suite for GNOME Logs is very crucial to further increase it’s robustness. In the coming weeks, I will be discussing with my mentors (David King and Jonathan Kang) on the various test cases that need to be included in the test suite. I will be sharing details about it with you in the next blog post.

As always, let me know your feedback/suggestions about what you think of these new features of GNOME Logs in the comments 🙂

GNOME Logs: Event Compression

Hi Everyone,

In this blog post, I am going to elaborate upon the event compression feature in GNOME Logs as implemented by me during last three weeks. It’s been exciting three weeks for me to hack on Logs code base and bring this crucial usability feature to life. First, let me speak in pictures about how this implemented feature looks like.

A GNOME Logs window showing compressed events looks like below:

The rows which indicate numbers along side the event messages represent a group of compressed events which have been hidden in the events list. I would like to call such rows “header rows”.

Clicking on a header row, toggles the visibility of the compressed event rows represented by it in the events list:

Here, a header row representing seven compressed events is expanded. The header row stores the details of one of these compressed events to be shown in the events list while it hides them. This compressed event, whose details are to be shown in in the header row, is selected such that it maintains the timestamp sorting order specified by “sort-order” GSettings key.

To keep things simple initially, only adjacent events w.r.t timestamp are checked for the compression/similarity criteria. The compression/similarity criteria for grouping these adjacent events under a header row is as follows:

1. Events containing messages whose first word is same (includes exact duplicates)
2. Events which have been generated by the same process.

This can be extended in future to compress non-adjacent events in specific batch sizes and accordingly group them under a common header row.

When we click on any row (including the expanded compressed events) except the header row, the detailed information regarding the event is shown in a fixed size popover:

Longer detail fields like “Message” are wrapped over multiple lines so that they do not exceed the width of the popover. All the detail fields which were available in the previous detailed event view are available in the new popover as well.

Well , what about event rows containing super long messages like coredumps or stacktraces ? Such event details are handled gracefully in the popover using a GtkScrolledWindow.

For example,

I will now try to explain in simple words about what exactly is going on behind the scenes. While fetching the events from the journal using sd-journal API in the model array  (backend), we count the number of adjacent events satisfying the earlier mentioned compression/similarity criteria. If the detected number of compressed entries in a group is 2 or exceeds 2, then a dummy event representing these compressed entries is added to the model array. This dummy event is nothing but our “header row”. This information is then parsed in the frontend and accordingly the events marked as compressed in the model array are hidden in the events list. You can follow progress on merging of this enhancement here.

Moreover, I have now started working on my next task which is writing a shell search provider for GNOME Logs. During previous year, I came out with patches containing a very basic implementation of shell search provider. I will now be improving on those patches according to review given by David. Currently, I am researching on various possible approaches to fetch the events from the model in the shell search provider. Further progress regarding this can be tracked here.

I am very happy to tell you that I will be attending GUADEC 2017 in Manchester. Many thanks to the GNOME Foundation for sponsoring travel as well as accommodation for me. I look forward to meeting all of you at GUADEC 2017 🙂

GSoC 2017 : GNOME Logs

Hello GNOME’ers,

Hope all of you are doing well 🙂

In this blog post, I want to introduce you to my GSoC project on GNOME Logs for this year. Last year, GNOME Logs saw the addition of a search popover and many improvements to it’s backend in terms of searching functionality. Moreover, some initial work regarding a Shell search provider for GNOME Logs was also done. I would also like to tell you that I was the GSoC 2016 intern on GNOME Logs and I must say, it was one of my summers spent well ! I got acquainted with the vibrant and diverse GNOME community and and on the other hand improved my skills in open source software development too , due to the valuable guidance given by my mentor, David King. This year, I will be mentored by David King and Jonathan Kang, two awesome people who are also the maintainers of GNOME Logs.

Regarding this year improvements to GNOME Logs, I will be working on the following tasks:

1. Move sorting by timestamp functionality from the frontend to backend.
2. Compress similar logs shown in the event list.
3. Write a shell search provider for GNOME Logs so that search results can be exposed to GNOME Shell.
4. Write unit tests for testing the user interface and search functionalities.

Here, I will be briefly explaining you about the first two tasks and the progress done on them. The first task is mostly about transferring the computational overhead to the model which is the actual backend in GNOME Logs. Currently, the list of events shown in the user interface are sorted w.r.t timestamp by GtkListBox using GtkListBoxSortFunc. The main motive of moving the sorting functionality to model is to reduce the complexity from the Logs frontend and make it simple. A challenging task here was to get the GListModel interface (which is implemented by GNOME Logs model class) to return the entries filled in the model array in a reverse order. It took me almost six months to figure out a correct approach for doing so. It would not have been possible without the guidance from my mentors and Lars Karlitski. Progress regarding the enhancement can be tracked on this bugzilla bug.

The second task is related to improvements in user interface and crucial for GNOME Logs from usability point of view. Currently, many adjacent events shown in the events list are either exact copy of each other or they are from the same process which logs some unnecessary events. This clutters the event list which makes it hard for a user to see the events which are of prime importance to him. So, the aim of this task is to compress these similar adjacent events. Allan Day of GNOME Design team has developed an excellent mockup to present the compressed events in an intuitive way. I would like to post his mockup here:

To keep things simple initially, adjacent events will be compressed if the first word in their respective messages is same which takes care of exact adjacent duplicates too. This similarity criteria will of course be extended in the future to include other types of similar messages.

As of now, I have implemented some part of the mockup except the popover. A video of the working prototype can be seen here. I will be ready with the patches in the coming weeks and further progress regarding the event compression enhancement can be tracked on this bugzilla bug.

That’s all for now. Let me know in the comments what you think of this year’s GSoC improvements to GNOME Logs. We will meet again soon.

Regards.

GNOME Logs Search Provider

Hello everyone,

In this post I will be telling you about the search provider for the GNOME Logs as implemented by me in the last two weeks.

When we type a search term in the GNOME Shell overview search bar, the top 3 journal entries relevant to the search term are shown as results:

As you can see in the picture above, we show the journal entry message and also the process name which sent the message to the journal in the description of each of the result.

When we click on a search result, the details of the result are shown in a newly opened window:

If we click on the application icon on left side of the results in the overview, a new window is opened which shows all of the results related to the entered search term in the overview:

You can see a working video of the search provider here.

So, as you can see above, GNOME Logs is the new GNOME Application joining the league of other GNOME Apps search providers for GNOME Shell. 🙂

By default the search term is searched in all the available journal parameters in GNOME Logs. This was all about the GNOME Logs search provider in it’s current state of implementation. You can track the further progress on search provider in my github branch.

Along with the search provider, I also worked to get the patches related to the search popover merged in the GNOME Logs repository. Progress on this can be tracked on this bugzilla bug.

Apart from this, a sad announcement. I will not be able to attend GUADEC this year due to my university project evaluations clashing with the schedule of GUADEC. I will be missing GUADEC and the amazing GNOME Community very much 😦

Feel free to comment about what you think of the GNOME Logs search provider.

Stay tuned for my upcoming final GSoC project report.

See ya !

GNOME Logs GSoC Progress Report

Hello everyone,

I will be mentioning all the progress done in the GNOME Logs GSoC project in the previous weeks. The search popover is completed, but the patches related to it are yet to be merged, which I hope, will get merged in the coming week. In this post, I will be telling you about features implemented by me in search popover. If you want a brief recap about the search popover, please see my earlier blog post about it.

The implemented search popover looks like this when the drop down button besides the search bar is clicked :

Clicking on the button related to what label, it opens up this treeview which allows us to select the journal parameters to search the user entered text from:

       

If we select any particular field from the treeview, it shows an option to select the search type:

The search type option is hidden when “All Available Fields” is selected as exact search type doesn’t make sense in that case. So, only substring search type is available by default in that case.

Clicking on the button related to when label shows this treeview from where we select the journal range filters:

If we select the “Set Custom Range” option, we go to another submenu:

It allows us to set a custom range for the journal, including the starting date time and ending date time. Clicking on the either of the select date button shows this calendar from which we can select the date:

Clicking on either of the select time button shows these time selection spinboxes:

12-hour format

24-hour format

The time spinboxes change depending upon the time format used in the current timezone.

This was all about the search popover as it is currently implemented by me. From next week, I will be working on writing a search provider for GNOME Logs which can be used to query journal entries according to the user entered text in the GNOME Shell search bar. Also, I will be working on getting the patches for search popover merge into the master branch of GNOME Logs. After all the patches related to search popover get merged, I will be making a video on how the search popover works. I would like to thank my mentor David King for guiding me in the right direction and helping me get my patches reviewed and merged.

My next post will be about the search provider for GNOME Logs coming next week.

So, stay tuned and Happy Coding 🙂

GNOME Logs search popover

Hello Everyone,

It’s been almost two weeks since my last post. In this post, I will be mentioning all the progress done in these two weeks. First of all, the searching part is now moved from the front end part to the actual back end of GNOME Logs which is the sd-journal API. So, now when the user types text into the search bar, the results are actually fetched from the journal itself. Progress on the project can be checked out on my Github repository. I would like to thank my mentor David King a.k.a amigadave for reviewing my commits and guiding me in the right direction.

GNOME Logs got a new search popover. I have implemented the GUI bits of the search popover for GNOME Logs.We decided to use a complete popover design as it feels modern and didn’t want the user to open any dialog. So, all the actions that the user can do will be contained within the popover itself. The popover design is inspired by the Nautilus search popover with much of design and code reused from it. It is coded by Georges Stavracas a.k.a feaneron and designed by Allan Day a.k.a aday. I would like to thank both of them because due to their work, I was able to implement the search popover much faster and also I learnt a lot about dealing with popovers by reading Nautilus search popover code.

I will give here a short image walk through on the search popover:

This is how the entire popover looks when clicked on the drop down button besides the search bar:

Clicking on the button related to “What” label  opens up a listbox from which we can select the search parameter for querying the entries from journal:

Clicking on the button related to “When” label opens up a listbox from which we can select the range of journal entries which will be displayed:

Furthermore, if we click on “Set Custom Range” , we go to another submenu in which we can select the date range of the journal between which the entries will be displayed. This is similar to the –since and –until options of journalctl command line tool. Currently, here we don’t allow entering the time as GTK+ does not have a specialized time widget but it can be done in future once GTK+ gets a time widget.

If we click any of the buttons, a text entry and a calendar is displayed from which the user can select the date or type the date into the text entry.

Clicking on “Back” button goes back to main menu.

The last option “Search Type” consists of two radio buttons which specify whether the user wants a substring match or an exact match for his query.

You can watch the video of search popover in action here.

In the coming weeks, I will be implementing all the back end functionality of the search popover (so that selecting the options in it actually affect the displayed results).

Let me know in the comments about what you think of the search popover design and if some options are missing from the current design or it needs some different options.

In the next post, most probably, I will be posting a link to a video where I will demonstrate a fully implemented search popover.

Till then,

Happy Coding 🙂

GSoC 2016 Introduction

Hi everyone,

My name is Pranav Ganorkar and I am a third year student at V.E.S. Institute of Technology, Mumbai pursuing Bachelor’s degree in Computer Engineering.

My very first introduction to Linux was in eighth grade when I booted the Live CD (Yes, CD’s used to be much popular at that time due to low penetration of Internet) of Fedora 13 on my Desktop. I was very much impressed by it’s user interface and started hacking on it. It was my first introduction to GNOME Desktop. At that time , in high school, I was mostly interested in installing various distros and trying out new desktop environments. What a wonderful life back then , wasn’t it ? 🙂

I started following GNOME Development when I was in my first year of Engineering. It was when I learnt about the GTK+ Toolkit and tried to make some simple applications using it.  It’s been almost three years now since I have been using GNOME as my primary desktop environment on Arch Linux. I started following GNOME Logs development from January 2016, got acquainted with it’s source code, started hacking on it and was able to make my first contribution in March 2016 with a lot of help from David, my mentor.

Now, some details regarding my GSoC project. I will be working on improving the search feature in GNOME Logs under the mentorship of David King. Currently, searching in GNOME Logs is only restricted to the front end part. Meaning, the actual search results are not fetched from sd-journal API (the API which GNOME Logs uses to fetch the entries from the journal) but using the filter function for GtkListBox in the already fetched results. So, my first task is to move the searching to the back-end part. That is, when the user types text in the search bar, the results will actually be fetched from the sd-journal API.

I will also be improving the GUI for the search interface. Currently, GNOME Logs does not have any option to graphically restrict the journal fields by which user entered text will be filtered. So, I will be adding a drop down menu besides the search bar to let the user select the various journal fields by which to restrict the search results from. Currently I have done a GUI mockup for it which looks like this:

Also, the advanced options menu button will provide the user with a dialog box which will provide with advanced journalctl filtering options like “since” and “until”. Also, user can specify the type of search : exact or substring which he wants to perform. Currently, the GUI mockup for advanced search dialog box looks like this:

Along with the above tasks, I will also be writing a search provider for GNOME Logs. So, GNOME Shell will be getting a new search provider. Yay 🙂

Finally, testing will be done to test all the above implemented functionalities. I hope that after the completion of my project, GNOME Logs will be able to match up with journalctl tool in terms of searching and filtering functionality.This is a short summary of my GSoC project.

Looking forward to an exciting summer and learning a lot in process about GNOME Development and it’s vibrant community 🙂

Also, feel free to comment about what you think of the current GUI mockups.

See you in next post. Until then,

Happy Coding !