Widely Vision

Quran Tracks Re-Namer

Posted by aboelnour on March 21, 2012


SA

Every one faces a problem after downloading Quran tracks, the tracks named in the sura number not in the sura name.

so this ruby script for any one want to convert the Quran track names from numbers to number+name.

this script takes as a parameter the Quran Folder and convert tracks names from “001.mp3″ to “001-الفاتحة.mp3″ and so on.

the script need a file contains the suwar names called names located in the same folder attached here also.

to run:

./renamer.rb /path/to/Quran/folder

you can Download the script file from here and the names file from here

and you can Download The Quran from here.

Best Wishes,

aboelnour.

Posted in Uncategorized | Leave a Comment »

Google Buzz Analyzer

Posted by aboelnour on October 28, 2011


Peace bo upon you,

In last days Google announced that they will shut down some of their products among of this products was Google Buzz.

As a Buzz Geek this news was shocked to me, but Google offered to Download your buzz’s to your computer throw Google takout.

after I Downloaded my buzz data, I was trying to know how many of posts I created on buzz and number of like or comments per post.

So I created a simple script in ruby which calculates number of likes and comments ,top likers ,top comments and posts.

I’d like to share this script with other Buzz Geeks.

Features:

  • Showing numbers of likes and comments per follower.
  • Showing your top followers in likes and comments.
  • Showing numbers of likes and comments per month for posts.
  • Showing Top posts in likes and comments.
  • Nice and Reliable UI.
  • Offline Tool so it’s totally safe, you are not required to submit your data to a remote server.

Screen Shots (it’s from my Data):



You Can Download GoogleBuzzAnalyzer Tool from here

NOTE:

  1. You should read the “README” file first to install Dependencies.
  2. It’s working only on Linux Debian based Distro’s due to Dependencies, If you are using any other Distro or even Windows you are responsible on installing the Dependencies.
  3.  Tested on Firefox and Chrome only, I recommend to use Firefox.

waiting for your comments and feed back :)

[UPDATE] Google Buzz Analyzer is now on GitHub

regards.

Posted in Google Buzz, Ruby | 3 Comments »

Memory overcommit in Linux Memory management

Posted by aboelnour on September 16, 2011


What’s happening when a program request more memory on machine running OS with Linux kernel?

last days I get the answer.When you use malloc() to reserve more memory it has a return value.

from malloc() man page:

 malloc() returns a pointer to the allocated memory,

which is suitably aligned for any kind of variable.

On error,these functions return NULL.

NULL is returned when error occur in allocating the block of memory that include the system didn’t find enough space to allocate it to your program.

With Linux kernel by default malloc() always succeed. Linux deal with memory allocating like airline when sell tickets, it sells more tickets than they have actual seats, in the hopes that some of the passengers don’t show up.

Linux assuming you’re not “really” going to use all of the memory you just asked for. The malloc()‘s will continue to succeed, but not until you actually try to use the memory you allocated will the kernel “really” allocate it.

As an example, consider the fork() system call, which copies all of a process’s memory for the new child process. In fact, all it does is to mark the memory as “copy on write” and allow parent and child to share it. Should either change a page shared in this way, a true copy is made.

This called “Memory overcommit

“Memory overcommit is a Linux kernel feature that lets applications allocate more memory than is actually available. The idea behind this feature is that some applications allocate large amounts of memory “just in case”, but never actually use it. Thus, memory overcommit allows you to run more applications than actually fit in your memory, provided the applications don’t actually use the memory they’ve allocated. If they do, then the kernel terminates the application.”

To make things more clearer you can try this examples:

[WARNING] “This examples may cause harm to your computer if you don’t know how to rescue your system from process eating all of your memory don’t run it”

example 1: allocate memory without using

#include <stdio.h>
#include <stdlib.h>

int main (void) {
int n = 0;

while (1) {
if (malloc(1<<20) == NULL) {
printf(“malloc failure after %d MiB\n”, n);
return 0;
}
printf (“got %d MiB\n”, ++n);
}
}

example 2: allocate memory and actually touch it all.

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main (void) {
int n = 0;
char *p;

while (1) {
if ((p = malloc(1<<20)) == NULL) {
printf(“malloc failure after %d MiB\n”, n);
return 0;
}
memset (p, 0, (1<<20));
printf (“got %d MiB\n”, ++n);
}
}

example 3: first allocate, and use later.

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#define N       10000

int main (void) {
int i, n = 0;
char *pp[N];

for (n = 0; n < N; n++) {
pp[n] = malloc(1<<20);
if (pp[n] == NULL)
break;
}
printf(“malloc failure after %d MiB\n”, n);

for (i = 0; i < n; i++) {
memset (pp[i], 0, (1<<20));
printf(“%d\n”, i+1);
}

return 0;
}

In example 1 you will notice that the program will print smoothly without harming your computer and it will stop when the program reach your ram boundary and exit successfully.

Explanation:

the program doesn’t use the memory he allocate it,so Linux doesn’t care to allocate it.

——–

In example 2 you will notice that program make the computer print slower.

Explanation:

the program after allocating each block he use it, so Linux allocate it after each memset() so it takes alot of time to reach ram boundary.

——–

In example 3 you will notice that the program print quickly and when he reach the ram boundary the computer will be slower.

Explanation:

the program will allocate the whole block of memory after malloc() finishes.program will print smoothly and after printing “malloc failure after MiB” the computer will become slower.

———————————

So what’s happing when process request for more memory and there no free memory to allocate it for the process?

Linux start a job called OOM killer “Out Of Memory” killer, Its job is to kill processes and free up some memory. Getting it to kill the right processes has been an ongoing challenge, however. One person’s useless memory hog is another’s crucial application.

This challenge is like:

“An aircraft company discovered that it was cheaper to fly its planes with less fuel on board. The planes would be lighter and use less fuel and money was saved. On rare occasions however the amount of fuel was insufficient, and the plane would crash. This problem was solved by the engineers of the company by the development of a special OOF (out-of-fuel) mechanism. In emergency cases a passenger was selected and thrown out of the plane. (When necessary, the procedure was repeated.) A large body of theory was developed and many publications were devoted to the problem of properly selecting the victim to be ejected. Should the victim be chosen at random? Or should one choose the heaviest person? Or the oldest? Should passengers pay in order not to be ejected, so that the victim would be the poorest on board? And if for example the heaviest person was chosen, should there be a special exception in case that was the pilot? Should first class passengers be exempted? Now that the OOF mechanism existed, it would be activated every now and then, and eject passengers even when there was no fuel shortage. The engineers are still studying precisely how this malfunction is caused.”

———–

Refrences:

http://lwn.net/Articles/104179/

http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html

http://www.win.tue.nl/~aeb/linux/lk/lk-9.html

http://www.gnu.org/software/gnusound/Documentation/ar01s05.html#id2572059

———–

I hope that any one find this post useful. any questions or feedbacks are welcomed.

regards,

aboelnour

Posted in C/C++, linux | Leave a Comment »

How a linux geek play with google guitar

Posted by aboelnour on June 9, 2011


Hi,

I guess you saw the google guitar on google.com

I faced a problem when I try to run the clips which my friends posted on facebook and buzz Because I press keys very slow and I can’t know which clip is this due to the Guitar doesn’t response to the numbers keys which located at the side numpad it only response to the numbers key on the top which I’m not familiar with it. so i wrote this C code to handle my problem and simulate keys pressing :

#include <X11/Xlib.h>
#include <X11/keysym.h>
#include <X11/extensions/XTest.h>
#include <stdio.h>
#include <unistd.h>

int
main (int argc, char *argv[])
{
sleep(2);
Display *display;
unsigned int keycode;
display = XOpenDisplay(NULL);
char* temp = argv[1];

while(*temp != 0)
{
//    printf(“TEST # %d\n”,*temp); //to debug
keycode = XKeysymToKeycode(display, *temp); // get key code

XTestFakeKeyEvent(display, keycode, True, 0); // Generate regular key press
XTestFakeKeyEvent(display, keycode, False, 0);// Generate regular key release
XFlush(display);
temp++;
usleep(500000);
}
return 0;
}

I used the X11 library  to simulate key press and release.

to compile you should link with X11 and Xtst

gcc test.c -lX11 -lXtst

Executable file for linux platforms here.

you should run the code in this format

./test 12-34-454-123

more samples:

‎48-89-89-87-78-78-76-67-6765-4343

2-45-2-45-2-56-12-678-12-678

8786-8786-8786-7-7675-7675-7675-6-6564-6564-6564-5

144-144-345-64-678-654-6564-354

987-67-6-56-7-676-987-7789-7-00-989-878-987

585-3453-12345-975-597-785-23465-321

Now open google.com and your terminal and run the program inside you terminal and quickly make your browser on the top.

best wishes.

Posted in C/C++, linux | Leave a Comment »

Optimization in GCC(1)

Posted by aboelnour on June 17, 2010


peace be upon you:

We will talk about the GCC optimization of your code in the incoming posts isA.

At first we have to know what happing when we compile a source code to run it with GCC:

• Preprocessing (to expand Macros)
• Compilation (from source code to assembly language)
• Assembly (from assembly language to machine code)
• Linking (to create the final executable)

There is an addition step the GCC do it Called “Optimization”.Optimization is a complex process. For each high-level command in the source code there are usually many possible combinations of machine instructions that can be used to achieve the appropriate final result. The compiler must consider these possibilities and choose among them.

In general, the generated code differ from CPU to another because Each type of processor also has its own characteristics some CPU’s provide a large number of registers for holding intermediate results of calculations,others must store and fetch intermediate results from memory.

so there is some steps the GCC which follow it to optimize your code:

1) Source-level optimization:

This form of optimization used by GCC occurs at the source code level, and does not require any knowledge of the machine instructions. There are many source-level optimization techniques we will took two common types: common subexpression elimination and function inlining.

1.1) Common subexpression elimination:

One method of source-level optimization which is easy to understand involves computing an expression in the source code with fewer instructions,by reusing already-computed results. For example, the following assignment:

x = cos(v)*(1+sin(u/2)) + sin(w)*(1-sin(u/2))

can be rewritten with a temporary variable t to eliminate an unnecessary extra evaluation of the term sin(u/2):

t = sin(u/2)
x = cos(v)*(1+t) + sin(w)*(1-t)

Common subexpression elimination (CSE) is powerful, because it simultaneously increases
the speed and reduces the size of the code.

1.2) Function inlining:

Another type of source level optimization, called function inlining, increases the efficiency of frequently-called functions.

Whenever a function is used, a certain amount of extra time is required
for the CPU to carry out the call: storing the function arguments, jumping to the start of
the function ,executing the code, and then return to the original point of execution when the function call is complete. This additional work is referred to as function-call overhead.

Function inlining eliminates this overhead by replacing calls to a function
by the code of the function itself (known as placing the code in-line).

The following function sq(x) is a typical example of a function that
would benefit from being inlined. It computes x^2 :

double
sq (double x)
{
return x * x;
}

This function is small, so the overhead of calling it is comparable to the
time taken to execute the single multiplication carried out by the function
itself. If this function is used inside a loop, such as the one below, then
the function-call overhead would become big:

for (i = 0; i < 1000000; i++)
{
sum += sq (i + 0.5);
}

Optimization with inlining replaces the inner loop of the program with
the body of the function, giving the following code:

for (i = 0; i < 1000000; i++)
{
double t = (i + 0.5); /* temporary variable */
sum += t * t;
}

Eliminating the function call and performing the multiplication in-line allows the loop to run with maximum efficiency.

—————————-

This is the first part the second one we will know another steps in optimization isA.

regards

Posted in C/C++, GCC | Leave a Comment »

CRIME!!

Posted by aboelnour on June 1, 2010


More than 10 peace activities has been killed and more than 30 has been injured in the international water by the Israeli Navy Commandos.

According to the international law this is a piracy.

And yesterday America use the Vito to prevent a decision which will open an investigation in this CRIME!!

True History of Israel

“If we thought that instead of 200 Palestinian fatalities, 2,000 dead would put an end to the fighting at a stroke, we would use much more force….”

By: Ehud Barak, Israeli Prime Minister
Source: November 16, 2000, quoted in Associated Press

Posted in Gaza | Leave a Comment »

Deeper in Libraries With GCC(4)

Posted by aboelnour on February 19, 2010


peace be upon you:

After making your library you will need to edit it or just read it’s Contents.
—————————-
Listing Symbols Names in Object Files:
The nm utility can be used to list all the symbols defined in (or referenced from) an object file, a static archive library, or a shared library. If no file is named on the command line, the file name a.out is assumed

run:

nm libc.a

look the man page for more info about options.
——————-
Removing Unused Information from Object Files:
The strip utility removes the debugging symbol table information from the object file or files named on the command line. The object file can be a static library, a shared library, or a .o file produced by the compiler. Depending on how much debugging information has been included in the file, stripping can dramatically reduce the size of the file.

run:

strip main.o libglom.a

this command will strip all debugging information
from the object file main.o and all the object files in the library libglom.a

The strip utility replaces the existing file with the stripped version, so if you want to be able to restore the original unstripped versions, you will need to save the files before stripping them or use the -o option to produce the output in a different file.
——————-
Listing Shared Library Dependencies:
The ldd utility reads through the object files in the binary executable or shared library named on the command line and lists all the shared library dependencies.

For example,the following command lists the shared libraries used by the bash shell program on a Linux system:

ldd /bin/bash

output:

linux-gate.so.1 =>  (0xb8022000)
libncurses.so.5 => /lib/libncurses.so.5 (0xb7fc3000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7fbf000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e60000)
/lib/ld-linux.so.2 (0xb8008000)

this is the shared objects which the bash use it the ldd utility determine the library and it’s path on your hard.
—————————-

that’s the end of our series.I hope that any one find it helpful. Any question is welcomed.

regards,

aboelnour.

Posted in C/C++, linux | 2 Comments »

Deeper in Libraries With GCC(3)

Posted by aboelnour on February 19, 2010


peace be upon you:
In last Post we talk about how to deal with the dynamic library in this post isA we will talk about how to make your Dynamic library and how to configure it.
———————————————
Example:
We will make a library contain 2 functions one print “Hello” the second print a given String.
here is the Files:

/*sayhello.c*/
#include <stdio.h>

void sayhello()
{
printf(“Hello from a loaded function\n”);
}

/*saysomething.c*/
#include <stdio.h>
void saysomething(char *string)
{
printf(“%s\n”,string);
}

/*say.c*/
#include <dlfcn.h>
#include <stdlib.h>
#include <stdio.h>

int main(int argc,char *argv[])
{
void *handle;
char *error;
void (*sayhello)(void);
void (*saysomething)(char *);
handle = dlopen(“libsay.so”,RTLD_LAZY);
if(error = dlerror()) {
printf(“%s\n”,error);
exit(1);
}
sayhello = dlsym(handle,”sayhello”);
if(error = dlerror()) {
printf(“%s\n”,error);
exit(1);
}
saysomething = dlsym(handle,”saysomething”);
if(error = dlerror()) {
printf(“%s\n”,error);
exit(1);
}
sayhello();
saysomething(“This is something”);
dlclose(handle);
}

then run:

gcc -fpic -shared sayhello.c asaysomething.c -o libsay.so

to make your shaerd object.

then to build your prog:

gcc -ldl say.c

to run:

./a.out

this message will appear:

libsay.so: cannot open shared object file: No such file or directory

that’s mean that the Linker couldn’t find your library that’s lead to we have to know how the linker search for the libraries.
the Dynamic Linker search by default in the directories /lib    /usr/lib.
and there is an environment variable called LD_LIBRARY_PATH.
this var contain the paths which the DL will search throw it so you need to edit it with the path of the directory which contains the library.
run:

export LD_LIBRARY_PATH=”/home/fakeroot/Desktop”

the run

./a.out

you will notice that the program works.

Another important environment variable called LD_PRELOAD this var is very useful because the linker load any library locate in this var before loading any library he should load.
With this var you can override some libraries in Linux like the libraries which is Responsible of detecting the user’s id :D

————————-

Configuring the Search for Shared Libraries:
In this part we will talk about the ldconfig utility.
The ldconfig utility performs two fundamental functions dealing with shared libraries.
First, it creates links so that references to shared libraries are always to the latest version.
Second, it stores a complete list of the available shared libraries in the file /etc/ld.so.cache.

The ldconfig utility reads the file /etc/ld.so.conf, which is a list of
directories containing shared libraries, and uses these directory names (along with the directories /lib and /usr/lib) to locate the libraries to be linked and listed in /etc/ld.so.cache.
The following command will create all the new links necessary and generate
a new version of the file /etc/ld.so.cache (you must be root):

ldconfig -v

for more information:

man ldconfig

—————————-

In next post I will finish this series isA.

regards,

aboelnour

Posted in C/C++, linux | Leave a Comment »

Deeper in Libraries With GCC(2)

Posted by aboelnour on February 9, 2010


peace be upon you:

It is a long time since i wrote here. sorry for the few who following this blog :D

In last post we talked about libraries and how to make a static one.

today we will continue our series in the libraries.

———————————–
Example on static libraries:

libhello.c File:

#include <stdio.h>

void hello(void) {
printf(“Hello, library world.\n”);
}

libhello.h File:

void hello(void);

demo_use.c File:

#include “libhello.h”

int main(void) {
hello();
return 0;
}

then run in terminal:

gcc -Wall -g -c -o libhello-static.o libhello.c

to obtain the object file of the source file which we want to make it a library.

ar rcs libhello-static.a libhello-static.o

to create the library.

gcc -Wall -g -c demo_use.c -o demo_use.o

to obtain the object file of the source which use the libarary.

gcc -g -o demo_use_static demo_use.o -L. -lhello-static

to compile the object file with the libarary.

./demo_use_static

to run the program.

and here you can make your own static lib and deal with it.

——————————–

Dynamic Linking:

In the static Linking we locating the library in the linking time this operation done by the linker which called by the compiler.

But there is another technique in linking calling dynamic linking which locating the libraries in the Run Time.

Once a program has been linked to use shared libraries, it must be able to find the shared library when it runs. The libraries are located by name, not by directory.

The loading of the libraries in the run time done by the Operating System.

to make the explanation easy we will start with an Example:

  1. #include <stdlib.h>
  2. #include <stdio.h>
  3. #include <dlfcn.h>
  4. int main(int argc, char **argv) {
  5. void *handle;
  6. double (*cosine)(double);
  7. char *error;
  8. handle = dlopen (“/lib/libm.so.6″, RTLD_LAZY);
  9. if(error = dlerror()) {
  10. printf(“%s\n”,error);
  11. exit(1);
  12. }
  13. cosine = dlsym(handle, “cos”);
  14. if(error = dlerror()) {
  15. printf(“%s\n”,error);
  16. exit(1);
  17. }
  18. printf (“%f\n”, (*cosine)(2.0));
  19. dlclose(handle);
  20. }

In this Example we will calculate the cosine of (2.0) using a library called “libm.so.6″ located in /lib.

Explanation of the code:

1-2: We include the standard library and standard I/O library.

3: We include the header dlfcn.h which contain the functions: dlopen(),dlsym(),dlerror(),dlclose() we will explain them later.

5-7: Declaration of void-char pointer 6: Declaration of Function pointer.

8: here we call the dlopen() function which loads the shared library into memory (if it’s not already there) and returns a handle that can be used to address it the first parameter is the library and the second one is a flag  passing to the Operating System.

9-12 ,14-17 : Handle Errors the dlerror() function returns a descriptive string describing the error that occurred on the most recent call to any one of the other functions(dlopen(), dlsym(), dlclose()). The dlerror() function returns NULL if no error occurred.

13: Here we call the dlsym() which return the addresses of the functions the first parameter is the address of the library and the second parameter is the name of the function which you want to use it from the library .

18: here we get the value of the cosine(2) with the function which we get from the library which loaded in the memory.

19: here we call the dlclose() Functions that detaches the current program from the shared library. If no other programs are attached to it,the dynamic library is unloaded from memory.

Note: The flag used as the second argument on the call to dlopen() can be RTLD_NOW,which causes all the functions in the library to be loaded into memory and become immediately available. The other option is to specify RTLD_LAZY, which will delay the actual loading of each function until it is referenced on a call to dlsym(). Either of these flags can be OR‘ed with RTLD_GLOBAL, which allows any external references in this library to be resolved by calling functions found in other (also loaded) dynamic libraries.

to build your program run:

gcc test.c -ldl -o testth

then to run:

./test

the option -ldl indicate that this program will need the Dynamic linker of the OS.

———————————–

so here we are we will cover in the next part how to make your own dynamic library and how to deal with the Environment variables and how to configure your library.

regards,

aboelnour :)

Posted in C/C++, linux | 3 Comments »

Deeper in Libraries With GCC(1)

Posted by aboelnour on November 29, 2009


peace upon you:
when you open a new file to write some C code you must to include some headers and libraries.
So in this article we will talk about this headers and libraries and how to deal with it throw the gcc with the terminal.
————————————————————-
Any Compiler consist of many parts one of this part is the linker.

To know what we mean with the linker we must know some basics about compiling stages.
when you call the compiler to compile some code it first turn your code to a thing called object file.
this object file contains the machine code which will run your machine.

you can get the object file by this

gcc -c main.c

you will notice that a file called main.o appeared.
In this stage the linker start working it takes this object file and link it to the libraries and headers which the program use it and produce an execution file ready to execute.
you can to call the linker of gcc to deal with the object file like this

gcc main.o -o a.out
This will produce an execution version of your program called a.out you can run it:

./a.out

you can merge more than one source file in one object file:

gcc main.o inlet.o outlet.o genspru.o -o spinout

Object Files in a Static Library:

Object files can be stored in a static library and linked from there in much the same way as they can be linked from separate files, except the linker will automatically search through the contents of the library and include only the object files that are necessary.

If nothing in an object file is referenced from inside the program, it is not included as part
of the executable.

A static library containing object files is known as an archive file, and it’s constructed
and maintained by a utility named ar. The name of an archive file normally has a
prefix of lib and a suffix of .a.the following sequence of commands compiles three
object files and stores a copy of them in a library named libspin.a. Then the linker
uses the object file named main.o and the contents of the library to construct an executable program named spinner:

$ gcc -c inlet.c outlet.c genspru.c
$ ar -r libspin.a inlet.o outlet.o genspru.o
$ gcc main.c libspin.a -o spinner

The first gcc command produces the three object files that are inserted into the static
library by the ar command. The last command compiles main.c into main.o and then
invokes the linker, which reads the contents of libspin.a to try to resolve external
function and data references made in main.o. A module stored in libspin.a is
included as part of the final executable file only if it contains a function or data item
referred to from a module that has already been included as part of the executable.

Inside the static library, along with the object modules, is an index that lists all the
names of global data and functions defined in the library. The linker uses this index to
determine which modules to include and which ones to ignore. Normally, this index is
created by the ar utility when the library is created or updated, but options are available
on the ar utility that can suppress the creation of the index. This can be useful when
maintaining a large library—multiple changes can be made without bothering to update
the index until the modifications have been completed. To create an index or to update
an existing index, you can use the ranlib utility. For example, the following pair of
commands use the -q option of ar to quickly append files to an existing archive without
updating the index, and then it uses ranlib to update the index to reflect the current
status of the archive:

$ ar -q lib.a file1.o file2.o file3.o
$ ranlib lib.a

The order of appearance of the modules in the library can make a difference. If the
same symbol is defined in more than one module, then the linker will find and include
the first module if it is looking for that symbol.
————————————————————-
In this article we talked about the static libraries.
In comming articles isa we will talk about the dynamic libraries and how to deal with it.
any feedback is welcomed
best wishes,
aboelnour.

Posted in C/C++, linux | Leave a Comment »

 
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: