Idbg Program Suite

Component and Usage Overview:

  1. Program idbg_scan
  2. Program mdbg
  3. Program idbg
  4. Program rdbg
  5. 'mute_flag' Usage   (Debugging Ruby Internally!)
  6. Debug Function Library
  7. Debug Library usage with 'eval.c' example
  8. Target File itst

This document describes the Idbg Debug Suite.   It provides the ability to install debugging commands into a 'C' Program at both function boundaries and specific line numbers.   It also provides a support for inserting debug support functions directly into the target program.

This program suite was originally developed to investigate the internals of the Ruby Interpreter, which is written in 'C'.   While this suite can be used with any 'C' program, there are several features that where written especially for interacting with Ruby.   Additionally, the examples showing modifications to version.c  and eval.c  are for using these Ruby-specific features.

NOTE: The file debug.c  has been modified to correct a rather large error! The functions mute_enable()  mute_disable() were inverted. This has been corrected!

NOTE: More information has been added explaining debugging Ruby Internally.


   Component and Usage Overview

The Idbg Suite consists for the following program components:
     idbg_scan      Used by 'mdbg' to create the Function Reference List
     mdbg	   Creates the Extension File which controls 'idbg' processing
     idbg	   Inserts the Debug Information into a 'C' Program
     rdbg 	   Removes Debug Information from a target file.

The rest of the manual describes the usage of the components and debug library.

1. Program idbg_scan

  Usage: idbg_scan <target_file>>STDOUT

    Where <target_file> is the 'C' File being modified.

Idbg_scan is a modification of an implementation ctags.   It generates a Function Reference List consisting of a List of Function Names and the Line Number of the first executable statement.

The Reference List is sent to STDOUT and has the following structure:

     function_1     32
     function_2     48
        :   :       :	   
     function_n     xx	    

2. Program mdbg

  Usage:  mdbg [-tag] <target_file> <cond_flag> >STDOUT

    Where -tag switch generates a ##TAG Line for each Code Block
    When <cond_flag> is present it's value is used to set the
       <cond_flag> for the generated ##LABEL Command!

Mdbg program creates a skeleton Extension File.   Mdbg executes idbg_scan internally for the target file and writes the results to intermediate file  '.idbg_tmp'.   Using this file it generates an Extension File.   It supplies some front-end boiler plate and a Code Block for each Entry in the Reference File.   The result is written to STDOUT.

The program provides two ways to modify the structure of Code Blocks.   A Code Block consists of a ##LABEL Command with an optional Conditional Flag, an optional ##TAG command, any user specified commands and an ##END Command. The -tag switch will cause the ##TAG Command to be generated and providing a <cond_flag> program argument will append the <cond_flag> value to the ##LABEL Command.

The Extension File Structure is as follows;

     ##FILE itst.c_org                 - The Target File Name
     ##DUMP                            - Causes Dump of the Command List
     ##BLOCK                           - Causes Dump of Prepend Block
     ##COM                             - Example of a Comment Block
     Comment Block
     ##CEND
     ##LABEL function1 8 cond_flag     - A Function-Name type Code Block with both optional entries
     ##TAG
     ##END
     ##LABEL ~~Marker_X1 96 cond_flag  - A Label-Name type Code Block with both optional entries
     ##TAG
     ##END

'function1'   Code Block generates the tag line:           "<itst.c> function1(...) at (8)"
'~~Marker_X1'   Code Block generates the tag line:    "<itst.c> (96) Marker_X1"

Notice:  When the program name ends in "_org" it will be stripped off before prefixing the label line.
      (It is assumed that somefile.x_org  is the base file for the modified file somefile.x.

The <cond_flag> will cause the generated result of the Code Block to be enclosed in a "if (cond_flag != 0) {    ...   }" Structure.  
The two tildes(~~)  are only used to indicate a different generation structure and are stripped from the Label Text.

3. Program idbg

  Usage:  idbg <target_file> <extension_file>  >STDOUT

    Where <target_file> is the base file to be modified
    Where <Extension_file> directs the modification process

The Extension File Contains the following Command Structures::

     ##FILE <filename> ... ##EOF         	  - Encloses Extension Commands for a 
                                         	    specific target file. If an Extension
                                         	    file contains no ##FILE commands, then
                                         	    all the commands in the Extension File
                                         	    will be applied against the target file.

     ##COM[MENT] ...  ##CEND        	          - Enclosed Block of Comments - Will be ignored
     //                                  	  - Line Comments - Ignored until 'End of Line'

     ##MOD[ULE] <filename>               	  - File to be included in the 'Prepend Block'
                                         	    The Prepend Block is inserted BEFORE the first
                                         	    line of the target file.

     ##DUMP                              	  - Will Print the Command List to be applied to
                                         	    the current target file. The list has been
                                         	    stripped of all ##FILE, ##EOF, ##COMMENT, ##CEND, 
                                         	    ##MODULE, ##DUMP, and ##BLOCK Commands.

     ##BLOCK                             	  - Will print the contents of the Prepend Block.
                                         	    
     ##LABEL <Text> <line #> [Flag]  ...  ##END   - Describes a Code Block to be inserted at the
                                         	    specified Line Number. The Code Block may contain
                                         	    one ##TAG Command and any number of user code
                                         	    lines. The ##END Command terminates the insertion
                                         	    of the Code Block

                                         	    If the [flag] option is specified, the code block
                                         	    will be enclosed by "if (flag != 0) { ... }"
                                         	   
                                         	    If the '$mute_mode' global is 'true' then all Code
                                         	    blocks will be enclosed by "if (mute_flag != 0) { ... }"

     ##TAG [flag]                         	  - Inserts one of two types tag lines. The first
                                         	    type is used by the 'mdbg' program to tag functions.
                                         	    Any Label Text NOT preceded by two tildes(~~) will
                                         	    be printed as "<label_text>(...) at ( <line_number> )".

                                         	    If the Label text is prefixed by two tildes(~~), the
                                         	    tilde characters will be stripped from the label and
                                         	    will be printed as "( <line_number> ) <label_text>".

                                         	    The ##TAG line can be inserted anywhere within the
                                         	    Code Block..

                                         	    If the [flag] option is specified, the Tag Line
                                         	    will be enclosed by "if (flag != 0) { ... }"


The Structure of the Modified Program looks as follows:

     ---------- Prepend Block ----------
     #define ENABLE_DBG
     #ifdef ENABLE_DBG
       Module Code
     #endif
     ---------- Target Program ------------
     program code            
      ;      :
     function (....
     {
     #ifdef ENABLE_DBG
       user code
       printf("function at nn");         -- Generated by ##TAG Command       
       user code 
       user code
     #endif
       function code 
          :   :
     }


The Extension File fragment used to generate the function insertion above:

     ::
    ##LABEL function nn
     user code
    ##TAG
     user code
     user code
    ##END
     ::
 

4. Program rdbg

  Usage:  rdbg <target_file>  >STDOUT

    Where <target_file> is the file modified by idbg

The rdbg program removes all the debug statements inserted by idbg. While it appears to work well, the user may be able to confuse it with complex preprocessor commands. It is recommended that the original <target_file> be saved rather than depending on rdbg.

5. 'mute_flag'  Usage

The global variable mute_flag  suppresses reporting during Ruby Initialization.   If the debug function library is NOT being used, then the following is required.
  1. Define int mute_flag = 0;  in ruby.h
  2. In each file that is processed by idbg  include extern int mute_flag;
  3. Include mute_flag  as the <cond_flag> in the mdbg command string.

If the debug library is being used, The modifications to Ruby are shown in section 7.

Example:  mdbg [-tag] <target_file> mute_flag >STDOUT 

In either case the debugging output is suppressed until the mute_flag  is set to one(1).   This can be done Ruby internally by calling mute_enable(). If you have included the debug code and all the modifications in section 7  you may include the methods mute_enable  and mute_disable  in your ruby programs!

6. Debug Function library

This library contains functions to print objects both singularly and recursively.   It also provides functions for muting output during Ruby Initialization.  This library is primarily used for examining Ruby Internals.   However, it is an example of how the Idbg Tool Suite may be used.   The Files provide are:

     debug.c
     debug.h

The following mute functions are provided to suppress ruby function reporting until initialization is complete.   This feature is used to support 'idbg' usage within ruby itself.   Using 'idbg' on eval.c example below uses this feature.  For this feature to work, the appropriate modifications to your version of Ruby must be made.

     mute_disable()     - Sets 'mute_flag = 1' 
     mute_enable()      - Sets 'mute_flag = 0'

The debug support functions provided are:

     set_debug_mode(flag, mode)     - flag sets the global 'debug_loaded' 
                                    - mode sets the global 'ruby_debug_mode'
                                        DBG_X - Displays Object Nucleus and the
                                                primary Ruby meta and class Objects   
                                        DBG_N - Sets the debug loaded flag, BUT does
                                                not dump objects like DBG_X
                                    - Executes 'must_disable()' if it has been uncommented
                                      by the user(See section 7 below).
     fill_stop_obj()                - Preforms the display function for DBG_0 Flag above
     print_object(klass)            - Displays contents of specified object
     display_object(klass)          - Recursively displays Objects until a primary Ruby
                                      Meta or Class Object is found.
     dump_nucleus()                 - Dumps Object Nucleus (Everything below the primary
                                      Class Objects.

As shown in the example below, 'set_debug_mode' should be called in 'version.c'.   This function sets up the debug functions, display the stop list (Primary Meta and Class Objects) and displays the object nucleus.   It also Enables the Mute Flag (ie mute_flag = 1).   As shown in the example below this flag is used to control the display of function Calls.

7. Using the Debug Library to debug and/or explore Ruby

Since the Debug Library is used by several files, it must be linked into the Ruby Distribution..   Additionally, the 'set_debug_mode' is needed by version.c,  both to display it's settings and in this case disable the 'mute_flag' after Ruby Initialization is complete. The changes to the standard Ruby Distribution are shown here:

     File: common.mk                 - Insert 'debug' into the object list
         :        :       :
        OBJS	= array.$(OBJEXT) \
		bignum.$(OBJEXT) \
		class.$(OBJEXT) \
		compar.$(OBJEXT) \
                debug.$(OBJEXT) \        <=== Insert This!
		dir.$(OBJEXT) \
                  :     :

If you wish to enable and disable the mute_flag  from a ruby program, object.c  needs to be modified as follows and the debug package included:

     File: object.c                 - Insert 'include debug.h' and add ruby mute_flag commands
         :     :           :
       #include "ruby.h"
       #include "debug.h"                <=== Insert This!
       #include "st.h"
          :     :          :
    rb_define_private_method(rb_cModule, "method_undefined", rb_obj_dummy, 1);

    rb_define_method(rb_mKernel, "mute_enable", mute_enable, 0);     <=== Insert This!
    rb_define_method(rb_mKernel, "mute_disable", mute_disable, 0);   <=== Insert This!

    rb_define_method(rb_mKernel, "nil?", rb_false, 0);
          :     :          :

An example of using this feature is shown below:

     File: hello.rb                 - Example file for mute_enable  demonstration

#!/usr/bin/ruby
#
mute_enable               ## Enables Debug Code to execute
#
puts "Hello, Ruby World"

The following are the debug initializations that need to be put in version.c.

     File: version.c                 - Insert 'include debug.h' and 'set_debug_mode()' function
         :     :           :
       #include "ruby.h"
       #include "version.h"
       #include <stdio.h;>
       #include "debug.h"                <=== Insert This!
         :    :     :
         :    :     :
       ruby_show_version()
       {
           printf("ruby %s (%s) [%s]\n", RUBY_VERSION, RUBY_RELEASE_DATE, RUBY_PLATFORM);
           fflush(stdout);
       //                                <=== Insert This!
           set_debug_mode(1, DBG_X);     <=== Insert This -- If you want Objects Dumped!
           set_debug_mode(1, DBG_N);     <=== Insert This -- If you DO NOT want Objects Dumped!
           fflush(stdout);               <=== Insert This!
       }
         :    :     :

Copy the files 'hello.rb',  'debug.c' and debug.h' into the Ruby Base Directory.   Run './confiure' and 'make'.  Then install the 'ruby' executable in the appropriate area for your system.   Run the command 'ruby -v hello.rb' and the result should be as follows if ruby_debug_flag is DBG_X:

>ruby -v hello.rb

ruby 1.n.n (yyyy-mm-dd) [i686-linux]

Set Flags: debug_loaded = 1
Set Flags: ruby_debug_flag = 256

Stop Object List:
-----------------
metaclass_object------>0x00f6fddce4
  :      :             :         :
rb_cStruct------------>0x00f6fd4b44

=========================================================
               OBJECT NUCLEUS DUMP
=========================================================

***************** OBJECT NUCLEUS ******************

Init_Object ==> Dump of Object Nucleus
  :      :             :         :
RCLASS ====> super --------------->0x00f6fddce4  metaclass_object

=========================================================
              END OF OBJECT NUCLEUS DUMP
=========================================================


Hello, Ruby World

If you selected setting the ruby_debug_flag to DBG_N, then the output should be as follows:

>ruby -v hello.rb

ruby 1.n.n (yyyy-mm-dd) [i686-linux]

Set Flags: debug_loaded = 1
Set Flags: ruby_debug_flag = 0

Hello, Ruby World

Now to prepare for debugging within Ruby a small change to 'debug.c' is required.  The call to 'mute_disable()' must be uncommented!  This in conjunction with the modified 'eval.c' file will allow the printing of function call notices.

void
set_debug_mode(int flag, int mode)  // mode = Sets Debug Option Modes
{                                   // flag = Enables(1) / Disables (0) 'debug' code
  debug_loaded = flag;
  ruby_debug_flag = mode;
  fprintf(stdout,"\nSet Flags: debug_loaded = %d\nSet Flags: ruby_debug_flag = %d\n",
	  debug_loaded, ruby_debug_flag);
//  mute_disable();                      <== *** Uncomment This Line ***
  fill_stop_obj();

Finally, the programs 'mdbg' and 'idbg' are run to modify 'eval.c'.  Mke a copy of the original 'eval.c' and name it 'eval.c_org'.   Then do as shown below:

  mdbg -tag eval.c_org mute_flag >eval.ext
  idbg eval.c_org eval.ext >eval.c

  ./configure
  make
  
  ruby -v hello.rb 
The results will be similar to the following:
ruby -v hello.rb

ruby 1.n.n (yyyy-mm-dd) [i686-linux]

Set Flags: debug_loaded = 1
Set Flags: ruby_debug_flag = 1

Mute Disable: Sets 'mute_flag = 0'       <== This should now appear!!

Stop Object List:
-----------------
metaclass_object------>0x00f6fddce4
  :      :             :         :
rb_cStruct------------>0x00f6fd4b44

=========================================================
               OBJECT NUCLEUS DUMP
=========================================================

***************** OBJECT NUCLEUS ******************

Init_Object ==> Dump of Object Nucleus
  :      :             :         :
RCLASS ====> super --------------->0x00f6fddce4  metaclass_object

=========================================================
              END OF OBJECT NUCLEUS DUMP
=========================================================


<eval.c>  rb_thread_wait_fd(...) at (11987)        <== The Function Notices will be present!
<eval.c>  ruby_set_current_source(...) at (1274)
<eval.c>  rb_thread_wait_fd(...) at (11987)
<eval.c>  rb_thread_fd_close(...) at (11584)
<eval.c>  ruby_run(...) at (1822)
<eval.c>  ruby_exec(...) at (1808)
  :        :      :       :      :         :
  :        :      :       :      :         :
<eval.c>  ruby_finalize_1(...) at (1709)
<eval.c>  blk_free(...) at (9013)
<eval.c>  frame_free(...) at (8990)
<eval.c>  thread_free(...) at (11256)

8. Target file itst

Program itst is provided to perform a simple test this system.   Do the following:

     mdbg -tag itst.c_org >itst.ext
     idbg itst.c_org itst.ext >itst.c

     make itst

The output of 'itst' should be match the following:

>itst
main(...) at (84)

---------- Test IDBG Features -----------
function1(...) at (12)
function1 internal note!
function2(...) at (26)
function2 internal note!
function3(...) at (40)
function3 internal note!
function4(...) at (54)
function4 internal note!
function5(...) at (68)
function5 internal note!
>