GSoC 2014 “MusicBrainz support to EasyTAG” Final Report

Hello all,

These 3 months was an awesome experience for me. Working with my mentor David King was a hell lot of learning experience. Also, the experience at GNOME Foundation was great.

In this post I will give a tutorial about how to use MusicBrainz suuport for EasyTAG. Following features has been implemented in this summer:

  • MusicBrainz Dialog
  • Helpful Operations
  • Manual Search
  • Automatic Search
  • Search Selected Files
  • DiscID Search
  • ApplyTAGs

MusicBrainz Dialog: This is how the MusicBrainz Dialog looks:

MbDialog

You can see that it is a lot simpler than CDDB Dialog. There are buttons for all types of search and many other helpful operations buttons.

Helpful Operations: The TreeView in MusicBrainz Dialog is multi-selectable i.e. you can select more than one rows at a time. Following operations have been implemented:

  • Invert Selections: Select a row if it is not selected and vice-versa.
  • Select All: Select all rows.
  • Unselect All: Unselect all rows.
  • Toggle Red Lines: During the course of using MusicBrainz you could see that a row may be displayed in Red Color instead of Black Color, because this rows children have already been downloaded and it will be good to use that already downloaded data. With this Toggle Button you can toggle the state of display of these Red Lines.
  • Stop: Stop a downloading operation in between.
  • Fetch More Results: MusicBrainz doesn’t allow to download all data at a time, instead it will allow us to download only some amount of data. If you didn’t find your expected results in the downloaded data you can use this button to get additional data.
  • BreadCrumb: You can travel through the results using the breadcrumb widget.

Manual Search: Lets say you want to get some information about an Artist or Album or a Track. You can simple type the name of the entity you want to search in the given text box, select the type of entity from combo box and click on the search button. Then just wait and let MusicBrainz Dialog do the work. Here’s an example for it.

SearchSelected

In the above example we have searched for the Album “Waking Up”. What if you want to retrieve its Track? Simple, double click on the row containing you favourite result. Here’s what you will get:

WakingUpTracks

You can see that the BreadCrumb Widget has added a new button with name “Waking Up”. If you want to go back to the results just click on the button “Albums”.

WakingUpRed

Now you could see that one result is being displayed in red line. That is because its results has been already downloaded. Double clicking on it will show the already downloaded results. If you want to re-download the results you can click on Refresh.

Search Selected Files: This feature is helpful when you have a bunch of files and you don’t want to search for them individually. Just select a bunch of files and click on Search Selected Files. Then, MusicBrainz Dialog will group different files into different albums and search these albums over MusicBrainz server. After that you can click on any of the required Albums, view its Tracks and Apply Tags.

I have selected files. 2 albums are searched over MusicBrainz. Below are the results. You can get tracks of any of these albums.

SearchSelected

Tracks of Bridge to Terabithia:

SearchSelectedBridgeToTerabitha

Tracks of Do Your Thing:

SearchSelectedDoYourThing

Going back to the results of album:

SearchSelected2

DiscID Search: What if you have a Disc and want to get its metadata from MusicBrainz? This is where DiscID feature will help you. Insert a Disc into your drive and click on DiscID Search button. You will get all Albums in that Disc.

Here I have searched for DiscID “lwHl8fGzJyLXQR33ug60E8jhf4k-“.

DiscIDSearch

Double clicking on album “Praat geen Poep”:

DiscIDPatGreenPopTracks

 

Automatic Search: Sometimes a user may also require access to FreeDB Database. For this it was decided that”Automatic Search” feature would also be present in new MusicBrainz support. But don’t worry it don’t use FreeDB Servers to get the data. It uses MusicBrainz server to get the results for FreeDB ID. Here’s an example:

Select a few files, Open up the MusicBrainzDialog and click on Automatic Search. This is what I got:

AutomaticSearchResults

Double click on the required album to search it over MusicBrainz Server. And you get the results:

AutomaticSearchNoMoreLies

Now select your favourite No More Lies album and retrieve its tracks:

AutomaticSearchNoMoreLiesTracks

Apply TAGs : To apply tags to selected files you can just select the Tracks you want to apply their tags to. And click on “Apply Tag Changes” button. There are two ways in which tags can be applied:

  • The default way is applied n-th TAG to n-th File. In other words, Tag of first selected Track is applied to first selected File, Tag of 2nd selected Track to 2nd selected File etc.
  •  If you think that above way of applying tag may give you some mismatching. You can check the button “Match lines with the Levenshtien algorithm”. Then for each selected file a best tag among the selected tags will be discovered using http://en.wikipedia.org/wiki/Levenshtein_distance and the tags will be applied.

It may be possible that there are multiple TAGs associated with a Track. In that case MusicBrainz will ask you to pick up the most relevent TAG. In case you did Search Selected Files, then only those selected files will be consider whose Album name is same as the album name of Tag.

For example in the above Automatic Search example, just click on “Apply Tag Changes” button and MusicBrainz will take care of rest.

In all the above examples did you notice that Search Text field is already filled with some text? The reason if a file is already selected in BrowserList of EasyTAG then its album will be automatically added in Search text field. This is easy as you don’t need to take your hand toward keyboard. Select the file, open MusicBrainz dialog and click on search. Done!!

MusicBrainz support is very robust and will display any error if encountered during the operation. It will certainly help many users to get more accurate and better results.

Cheers!! 😀

Compiler Structure

This post is a follow up from my previous post. Here I will talk about parts of a compiler. A Compiler usually contains following parts:

  • Lexical Analyser (Lexer)
  • Syntax Analyser (Parser)
  • Semantic Analyser
  • Symbol Table
  • Intermediate Code Generator
  • Code Generator
  • Code Optimizer

The Source Code goes through the these phases and is finally converted to Target Language (which can be Machine Language or Byte Code).

flowchart2

Let us have a short talk about each one of them.

Lexical Analysis: This is the phase when the source code is converted into a stream of Tokens. These tokens are meaningful character strings. They would have some predefined meaning in the syntax of a language. For example, in C “int” could be converted to a Token called “Integer Type”. This token will then be used in the subsequent phases of the compiler which tells them that “int” describes “Integer Type”.

Syntax Analysis: This is the phase when the Source Code is analysed and converted into its Parse Tree or Syntax Tree. If any error is found in this phase then it is reported to the user. This phase takes the stream of Tokens from Lexical Analyser. Syntax Analyser is actually an Automata (which is a type of Turing Machine). It is created on the basis of the Syntax of Language. Syntax Analyser for C and of C++ are different. Most of the Syntax Analyser creates an Abstract Syntax Tree which actually depicts the structure of the source code.

Semantic Analysis: In this phase Compiler checks for Semantic errors of a language. Semantic errors mainly consists of type errors and whether variable has been declared or not. It makes a heavy use of Symbol Table. For every variable it encounters it gets variable’s type in Symbol Table and check with the language rules. It also performs Type Conversions based on the language rules. For example, the expression “1.5 + 2” will give a Float but “2” is an Integer. In this phase compiler will convert “2” to Float and apply float addition operator.

Symbol Table: This part of compiler is responsible for storing all the Type Information about all variables or functions. For example, if you have declared your variable named “foo” as an integer then Symbol Table will store it. This part is used by all the stages of compilers. It is mainly created at the Syntax Analysis stage.

Intermediate Code: Intermediate Code may or may not be present in a Compiler structure. It depends on the Compiler Designers whether they want the Intermediate Code to be generated or not. Intermediate Code can be of many forms like Abstract Syntax Tree, Three Address Code, Static Single Assignment etc.

Code Optimizer: Code Optimizers mainly work on Intermediate Code. They apply many algorithms for optimizing the code. Similar to Intermediate Code, Code Optimization is not a necessary part of compiler. Even many compilers like GCC or LLVM implements a command line switch to turn off/on and set level of Code Optimization.

Code Generation: Code Generation is the final step of Compiler. At this step the target language code is generated. It can be Byte Code or x86 Assembly Code.

In the next post I will start with Lexical Analysis phase of compiler.

Cheers 😀

Introduction to Compilers

Hey all,

I have been trying to create my own compiler for a simple language like C (by language I mean it don’t contains many construct, sure programming correctly in C is not easy :P) but I couldn’t find any good tutorial which go through every thing about creating compilers. And I don’t wanted to go through a 1000 pages Dragon Book (well who would :o). I found some other books too like Writing Compilers and Interpreters: A Software Engineering Approach, 3rd Edition but it concentrates on Top Down parsers instead of Bottom Up Parsers (don’t worry I will tell you what they mean). So, finally (Believe me it required so much of motivation) I decided to go through Dragon Book. And I was able to create a working compiler for a subset of C language for x86. The source code could be found here: CCompiler – GitHub. But I don’t think you would want to go through the source code because 1) The project is difficult to understand and 2) The way I wrote the code it is even more difficult to read it (After all it was just a test project).

Now, I am writing a new language named Leopard (I love Leopards 🙂 ). Its source code will be good for sure. You can find more about the code here Leopard – GitHub. It will run on a Virtual Machine and is Garbage Collected and JITed. In this and the following blogs I will write about how to write Compilers, all the problems I faced while writing the Compiler, Assembler, Virtual Machine etc. I will tell you a solution for them. I didn’t use any Compiler Writers like Lex or Yacc. Instead everything has been written by hand (that is the beauty of Compilers, it is hard isn’t it :D). Compiler and Assembler has been written in C#, Mono and Virtual Machine in Java (yes, I want to write it in C# but I want to try my hands over Java too, lets see if Java is any better than C# :P).

Pre-requisites for Compilers are so many. You must know Theory of Computation, many Data Structures, any Assembly language (I would prefer x86) and many others. But no one would want to learn all these right? (It is human nature isn’t it? Who wants to study). So, I will tell you about all these things. These are things I am going to write about:

  1. Introduction to Compilers, Interpreters, Virtual Machines
  2. Automata, Languages, Grammars.
  3. Lexer
  4. Parser
  5. Semantic Analyser
  6. Intermediate Code Generation
  7. Code Generation
  8. Assembler
  9. Virtual Machine
  10. Garbage Collection
  11. JIT

I know its too much, but if you want to write your own compiler you have to do that. So, at the end of this blog series you would have the knowledge of how to develop a Compiler, Assembler, Virtual Machine with JIT and Garbage Collection. We will not use LLVM or any other compiler infrastructure project instead we will start with scratch.

Lets start with Introduction to Compilers.

It is said that there are two types of languages in this world.

  • Compiled
  • Interpreted

Compiled Languages compiles directly to the machine code. C, C++, Objective-C are examples of compiled languages. The code written in C is compiled directly to machine code. This is the typical work flow of a program written in compiled languages.compile

Source files contains the code, Compiler compiles it into Object File. These object files are then linked to different libraries (if required) and an executable file is produced. Now this executable file contains only binary code. This binary code can be executed directly by the processor. Compiled code gives us the best speed as it is directly executed by processor. But we do not have any control over such compiled code. It can be malicious and if executed by the processor it could lead to havoc.

Interpreted Languages do not compiles to machine code in one go. Rather a program known as Interpreter will read a line of source code, checks if it is correct and then executes it. If it founds an error then the program will be stopped by the Interpreter. Examples of such languages are BASIC, Shell Scripting, Perl, Python, Ruby. Unlike Compiler, processor doesn’t executes the code directly. A line is first converted to code and then executed by the processor. Interpreted Languages gives worst speed.  But, we have a control over such code. Interpreter knows which code is going to be executed and it could check whether that code could do some malicious operations or not. If it does then it will stop it right there.

As you can see, we could have good security but on the cost of speed and good speed on the cost of security. In 1970s, LISP was introduced. It was the first language to bring something in between of Compiled and Interpreted Languages. Later with Java this concept became very popular. Here is the basic idea about it.

The source code is compiled into Byte Code. This Byte Code is then executed by a Virtual Machine.

java-compiler

This is the work flow of a Java Program. The execution of Byte Code by Virtual Machine can be done is two ways:

  • Interpreting the Byte Code: Virtual Machine will behave similarly as an Interpreter to Byte Code. It will read each instruction an execute it. The speed of a program executed now is much better than Interpreted Languages but less than Compiled Languages.
  • Just In Time Compilation: Instead of read and converting each instruction of Byte Code one at a time, Virtual Machine can compile some amount of Byte Code (may be one function or one compound statement block) instruction into machine code and save the compiled code for future use. Whenever it founds that the Byte Code instructions to be compiled have already been compiled it will reload the saved compiled code. This increases the speed significantly as compared to “Interpreting the Byte Code”. It is even seen that JIT has surpassed the performance of C/C++ Compiled code. This is because at runtime JIT has information about the current processor being used and it can then apply some optimizations specific to that processor. But Optimized C/C++ Code is faster than JIT. When we will be developing JIT we will talk more about it.

Not only Java but today many languages follow this approach. It includes C#, VB.NET, C++/CLI, Python, Ruby.

Next time we will see a block diagram of Compiler and how it works.

Cheers 😀

GSoC 2014 “MusicBrainz Support to EasyTAG” Progress Report : 14 July – 27 July

Hello All,

I wasn’t able to post blog last Sunday because of my tight schedule. In the past two weeks I have done the following things:

  • Completed Apply TAGs for Automatic Search. After Automatic Search, user can select the Tracks and then click on Apply TAGs. If user wants MusicBrainz to use Levenshtein Algorithm for finding the best possible Tag for the File, he can select the check box. Below is the screenshot:

 

apply

  • My mentor David King did a review of the code and I did all the changes.
  • I also wrote a test case for FreeDBID Search.

All the features which I talked about in my proposal has been added. So, this means the time is for some heavy testing. In the coming week I will be doing testing of each and every feature of MusicBrainz Dialog. Lets hope I do find some bugs. 😉

Thank You

GSoC 2014 “MusicBrainz support for EasyTAG” Progress Report: 7 July to 13 July

This week I completed Automatic Search. They way automatic search works is very similar to the way it worked previously. This is how it work:

  • Select a number of files related to one album in BrowserList.
  • Open MusicBrainz Dialog and click on Automatic Search.
  • Now, FreeDB ID will be calculated and it will be searched using MusicBrainz Web Service.
  • Results will be dispplayed.
  • Double click on any result to search it over MusicBrainz Database.

The way now FreeDB ID is calculated is exactly same as with CDDB.

Here is an example,

I have selected some files related to album and clicked on the Automatic Search. This is what I getautomatic0

Now double click on the Result.

automatic1

“Rubbish” has been searched over MusicBrainz. You can now double click on any album and view its Tracks.

This week I will implement Apply TAGs for Automatic Search.

GSoC 2014 “MusicBrainz Support for EasyTAG”: 30 May – 6 May 2014

Hello all,

This week I implemented “Apply TAGs” feature corresponding to “Scan Selected Files”. “Apply TAGs” uses DLM Algorithm to find the best matching Track to which the Tags will be applied. Here I will show a work flow of using MusicBrainz Dialog.

MusicBrainz Dialog

This is the MusicBrainz Dialog. Cool isn’t it? And it is going to become more cool, if you would like to search. Let us type “Waking Up” in the Search ComboBox and click on the Search button.

These are the results which I get:
Waking Up Results

To all who don’t know, Waking Up is the Album by One Republic. Did you see the ToggleButton with “Albums”, that is the BreadCrumb Widget. It will help you navigate in the search results.

Now lets double click over one of the Waking Up rows, to get all the Tracks.

Waking Up

The above image shows an intermediate state when the information about Tracks is being downloaded. See the “Stop” button is in “Enabled/Sensitive” state now. You can press the “Stop” button to stop the search. The final results are here:

Waking Up Tracks

You can see the Track Title and information associated with it. There are many other options available to help you see the information. You can see those options in the ToolBar below.

Here I showed you can example of Manual Search. Similarly you can do “DiscID” or “Scan Selected Files” search.

For “DiscID Search”, insert the compact disc and press this button. Then MusicBrainz will take care of everything.

For “Scan Selected Files” search, select a bunch of files in the EasyTAG’s BrowserList and press the button. From there you can select any album and press “Apply Tag Changes” to apply the tags to the selected files.

Next week I will implement Automatic Search.

Thank You

GSoC 2014 MusicBrainz Support to EasyTAG, Progress Report 23 June 2014 – 29 June 2014

Hello all,

I passed the Mid-Term Evaluations yaaay!!. Thanks to my mentor David King for so much all through the previous 2 months. I promise to deliver even better in the coming months.

Ok, so lets see what I have done in the past week.

  • I have done many bug fixes.
  • Implemented the Refresh Operation.
  • David and I decided to change the whole GUI making it more simpler. Replaced all the tabs with just buttons. That make the GUI really simple. And now user can do search with just one click, instead of two.
  • By default, the type of entity to be searched will be “Album”.

Currently, I am working on “Apply TAGs” feature and I hope it to complete it within a day or two and then will do some coding style improvements.

We got some problems with Automatic Search as it relies on FreeDB or CDDB search. Searching FreeDB database in MusicBrainz through MusicBranz FreeDB Gateway gives expected results but not through MusicBrainz WebService :(. So, we decided to leave Automatic Search. If possible I will convert current Automatic Search code to use MusicBrainz Web Service methods.

Thank You