C Implementation of Red Black Tree

Sample Output

kdt1

kdt2

C Implementation of K-dimensional Tree

Sample Output
Sample Output

kdt1

kdt2

C Implementation of LEFTIST Tree

C Implementation of LEFTIST Tree

Sample Output

left1

let2

left3

KVM guest reports errors – How to troubleshoot or track it ? systemtap ?

I have seen reports saying that – under some scenarios, the KVM guest system reports I/O errors
in its ‘dmesg’ against virtio block devices and also, some times it cause the filesystems inside the guest to go ‘READONLY’ mode.

Tracking such issues are bit difficult and its a pain. I believe we should make use of different
profiling tools to find out whats happening behind the scenes.

I like systemtap for such issues and I will describe a particular scenario in this article.

Below is one of the excerpt [where we are almost sure about the return code] which you may see in such situations.

While going through the ‘qemu-kvm’ code path, we can see 2 tail functions which can reveal or give some more
data on this error message.

One being ‘virtio_blk_handle_write()’ (caller) and other being ‘virtio_blk_rw_complete()’ (callee)

The virtio block request will be processed by backend functions and these functions are called at almost end of it.

Mostly when guest report i/o errors, the virtio_blk_rw_complete() would have received ‘EIO’ ( Error code : -5)
or an Error from virtio_blk_handle_write() function.

As you can see in the below code snippet 1, there are 2 places where the caller pass ‘EIO’ to the callee.

[snip 1]

So above will only pass “EIO” to the callee, its triggered as soon as the ‘request’ is invalid.

Lets see second part of the same func()

[snip 2]

I am more interested in blkreq[*num_writes].cb = virtio_blk_rw_complete;

This is the callback from the block layer, so I am interested to see the ‘error’ returned from it.

I have written a system tap script to track more information about this scenario when this issue happens.

For general availability, I have pushed this to my github account :

github.com/humblec/qemu-kvm-stap

The output would look similar to below:

As you can see above, the ‘Return Code Received” would be something other than ‘0’ when issue happens.

Rest of the fields are self explanatory I feel.

Please shoot your questions, if you have any..

Make sure you try this sytemtap, before applying in any critical servers/situations..
Also, play it with your own risk.. ūüôā

How-ever let me know if I can help you with..

ps# Thanks to ‘fche’ ( web.elastic.org/~fche/ ) for some systemtap foo.

linux system call error codes ( errnos) to human readable error strings..

 

 

Q) ¬† “I got an ‘errno’ returned from the system call and want to map to the exact error message or to the error string which is human readable..

How can I do that ??

This was answered quite few times, how-ever I forgot to share it here, but not this time.

Here is the complete listing of linux error code numbers with its explanation:

 

Why I pasted all these and From where I got it ?

1) ( Why )=> Ans: As a quick reference

2) ( From where) => ‘Errnos’ upto “35” is available in /usr/include/asm-generic/errno-base.h and >35 is available in /usr/include/asm-generic/errno.h

 

A quick start (beginners guide) on system tap and its usage!!

Yes, instead of pointing to urls which slowly conclude the things, I will give a quick start here:

Why a quick start/read:? 

At times it is not a good choice for some people¬† to go and read all the document and get it done, rather they would¬† like to know the basic syntax/usage and a quick overview, so the post ūüôā

The system tap scripts mainly composed with below notation:

probe <event> <handler>

Probes may be broadly classified into “synchronous” and “asynchronous”.¬† A “synchronous” event is deemed to occur when any processor exe‚Äź
cutes an instruction matched by the specification.¬† This gives these probes a reference point (instruction address) from which more¬† con‚Äź
textual data may be available.¬† Other families of probe points refer to “asynchronous” events such as timers/counters rolling over, where
there is no fixed reference point that is related.¬† Each probe point specification may match multiple locations (for example, using wild‚Äź
cards  or  aliases),  and  all them are then probed.  A probe declaration may also contain several comma-separated specifications, all of
which are probed.

EVENT can be any of the following type..
—————————————————————
kernel.function or kernel.statement
process.statement
timer.ms
begin
end
(tapset) aliases

HANDLER
—————————————————————–

filtering/conditionals ( if.. next)
control structures (foreach, while)
Helper functions: pid,execname, log

Below are examples of ‘valid’ probe points..

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† kernel.function(“foo”).return
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† process(“/bin/vi”).statement(0x2222)
              end
              syscall.*
              syscall.*.return.maxactive(10)
              sys**open
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† kernel.function(“no_such_function”) ?
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† module(“awol”).function(“no_such_function”) !
              signal.*? if (switch)
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† kprobe.function(“foo”)

For ex:

     To trace entry and exit from a function, use a pair of probes:
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† probe kernel.function(“sys_mkdir”) { println (“enter”) }
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† probe kernel.function(“sys_mkdir”).return { println (“exit”) }

Note, expressions like ‘*’, ‘?’ ..etc , those are totally valid and much helpful in certain scenarios.

HOW TO RUN:
———————————————
SystemTap scripts are run through the command ‘stap’. stap can run SystemTap scripts from standard input or from file. System tap have 5 level of execution.

Below is a list of commonly used stap options:

-v
Makes the output of the SystemTap session more verbose. You can repeat this option (for example, stap -vvv script.stp) to provide more details on the script’s execution. This option is particularly useful if you encounter any errors in running the script.

-o filename
Sends the standard output to file (filename).
-S size,count
Limit files to size megabytes and limit the the number of files kept around to count. The file names will have a sequence number suffix. This option implements logrotate operations for SystemTap.

-x process ID
Sets the SystemTap handler function target() to the specified process ID. For more information about target(), refer to SystemTap Functions.

-c ‘command’
Sets the SystemTap handler function target() to the specified command and runs the SystemTap instrumentation for the duration of the specified command. For more information about target(), refer to SystemTap Functions.

-e ‘script’
Use script string rather than a file as input for systemtap translator.

-F
Use SystemTap’s Flight recorder mode and make the script a background process. For more information about flight recorder mode, refer to Section 2.3.1, ‚ÄúSystemTap Flight Recorder Mode‚ÄĚ.

stap -e “script” -c “target program”
stap script.stp -c “target program”

Using “-p4” you can create a system tap module as shown below;

¬†¬†¬†¬†¬†¬†¬† $ stap -p4 -e ‘probe begin { printf(“Hello World!\n”); exit() }’
/home/user/.systemtap/cache/85/stap_8553d83f78c_265.ko

Run staprun with the pathname to the module as an argument.

        $ staprun /home/user/.systemtap/cache/stap_8553d83f78c_265.ko
Hello World!

Now if you want to know more , please refer #

sourceware.org/systemtap/SystemTap_Beginners_Guide/