Files in directories

I was playing with some rsync-ing nowdays and somehow there was more files than I was expecting. Tons of files... I was wondering how to find out where are those files in the huge directory structure.

My solution was this:

find -L . -maxdepth 1 -mindepth 1 -type d -exec /bin/bash -c 'echo -n "{} "; find -L {}|wc -l' \; | column -t

It prints the file count of the directoryies where you run.

With the help of this pice of .... hmmm... beauty, I was able to find thousands of files that neadn't to be there.:)

It works like follows:

stone@home:/usr/src/linux-headers-3.0.0-14$ find -L . -maxdepth 1 -mindepth 1 -type d -exec /bin/bash -c 'echo -n "{} "; find -L {}|wc -l' \; | column -t
./kernel         28
./arch           8032
./ubuntu         28
./Documentation  57
./virt           3
./drivers        1673
./tools          51
./ipc            2
./scripts        241
./crypto         6
./include        2670
./usr            3
./security       24
./sound          240
./net            194
./block          4
./firmware       2
./fs             221
./mm             4
./samples        19
./lib            21
./init           3
stone@home:/usr/src/linux-headers-3.0.0-14$ 

Goodby Unity, Wellcome Xfce (some tweaks to make me happy)

I was fed up with the Unity and Gnome3 thing. I wanted to get my Gnome2 back and the old just works feeling. So I switched to xubuntu...

I was happy after configuring the panels to mimic the old Gnome2 but had to do some other tweaks to get the old feeling.

First was the annoying habbit that windows jumped between workspaces (for example when clicking a link in Thunderbird made my Chrome to jump to the same workspace) but after a bit Goggling I found the solution:

xfconf-query -c xfwm4 -p /general/activate_action -s none

to get the old scrollbars do:

apt-get remove overlay-scrollbar liboverlay-scrollbar*

Now I am completly happy!:)

Bitcoin -- money of the future

Bitcoin is a P2P virtual money what is used as real money on the Internet for several years now. Some sais that this is the greatest innovation and the most dangerous thing since the born of the Internet. It is fully anonymous, decentralized and secure payment system. It doesn't needs a central authoritative to make it work and even governments can't control, tax or stop it. But for me the mathematics and the idea what makes it work is the greatest about it.

First of all it is backed up with RSA. So you have your private and public key (you have more actually to make it more private) and you can spend and receive money with them. Second it is backed up with a clever distributed time stamping solution to prevent double spendings.

Bitcoin as it is a chain of transaction blocks. This chain is maintained by the whole network and contains all transactions since the beginning (if it gets very long, there is ways to compress it). When you want to spend your money you concatenate the amount of money, the last blocks signature wih the recipient's public key and sign them with your private key then broadcast it on the network.

Clients do a proof-of-work process to include all new transactions in the chain. They form a block with the last block's signature, the new transactions and a special random part. This last random part is what is the prof-of-work. The client have to guess this part as to make the new block's hash (sha256) start with a fixed amount of zeros. Finding this kind of block is hard, the whole network can find just one in every ten minutes. When the machines become fast or the network grows the number of required zeros will grow.

So you have this chain of blocks with many zeros, does it good for you? Yes, because if your transaction is in the chain and is followed with several new blocks you can be sure that it is extreme hard to double spend that money what you got. Only an attacker who owns the majority of CPU power of the network can undo a transaction, and this scenario is very unlikely.

Every client works on his chain as it knows it and computes the proof of work. If a longer chain appears (some other client manages to do the work) it switches to it. Always the longest chain considered to be the real one. This is why the majority of CPU power rules the thing, because they can make the chain grow faster. An attacker's alternative chain will fall behind fast and gets ignored by the network and the first real spending remains the true.

How can someone get coins to spend? How Bitcoins are created? Since there is no central issuer Bitcoins materializes from thin air. There is a limited amount of Bitcoins can exists (21.000.000, but the smallest amount is 0.00000001 BTC, so inflation will take care that there will be enough for everyone) and till this amount is not created (the creation slows as time goes by, around 2100 will the last created) Bitcoins given to those who creates the proof-of-work. Those who does this coin creation seriously called the Miners. But Bitcoins can be exchanged from any ordinary money too (I think it can be a good investment as the price of Bitcoins rises).

As you can see the thing works. The details and the proofs for the method can be found in Satoshi Nakamoto's paper Bitcoin: A Peer-to-Peer Electronic Cash System. You can get the program from www.bitcoin.org and start using Bitcoins. You don't need to register, just run the program. It is available for Linux, OS-X and Windows. The Bitcoin users and accepting businesses number grows rapidly.

Rename Cyrus IMAP folders

I had the task to restructure some shared IMAP folders on our IMAP server running Cyrus.

I come up with the following solution.

1. Rename the IMAP folder in cyradmin with the renm fromwhat towhat command


2. Adjust all users subscribed folder list by issuing the following command in the /var/lib/cyrus/user directory:
   
   

   find . -iname '*.sub' -exec sed -i 's/fromwhat\x09/towhat\x09/' {} \;

   
   

This did the thing for me. Cheers!:)

Secure wireless gateway

I'll go to abroad in the near future. At the hotel there will be Internet connection at the room, but only wired. I'll take my Android phone and my laptop, so I'll bring a small wireless router with me too. It is a Fonera 2g as hardware but runs OpenWrt.

As I'll have my own AP there why not secure my connection? I also have a private server somewhere at the USA, so I fired up an OpenVPN connection between my router and my server, and redirected all data through it. So now I'll have unsniffable connection in the hotel.

How I did it? It was easy after putting the VPN together (was also wasn't a big deal). Here is the recipe:

First I configured the OpenVPN normally but added the redirect-gateway at the end of the client config file, to make it redirect the default gateway through the VPN. After that I enabled it at boot time with

/etc/init.d/openvpn enable

I've edited /etc/firewall.user and added the next 3 lines to it:

iptables -I forwarding_rule -i tun+ -o br-lan -d 192.168.6.0/24 -j ACCEPT
iptables -I forwarding_rule -i br-lan -o tun+ -s 192.168.6.0/24 -j ACCEPT
iptables -t nat -I POSTROUTING -o tun0 -j MASQUERADE

where 192.168.6.0/24 is my wireless IP network.

And that's all. If now I boot the router it automatically connects to VPN and after redirects the gateway through it.

How to block ads in Opera?

As far as I know (after 1 hours of goggling:)) I've found that the solution provided by http://www.fanboy.co.nz/adblock/opera/ is the best source for blocking ads in Opera. Just download http://www.fanboy.co.nz/adblock/opera/urlfilter.ini and put it on your Opera directory (described at the begining of the file) and voi la it blocks ads.

Adventures in native ZFS on Linux

I started using ZFS on my Linux box around 1 year ago, after I've taken part on a Sun training where BalaBit (my workplace) sent me. ZFS astonished me. It combined the whole RAID, LVM and file system thing together and made it more pleasant with constant data integration check, and powerful caching. I felt in love.

But wait, stop, what the hack is ZFS? It is a file system and a build in logical volume manager by Sun Microsystems. With many interesting features like: data integrity, support for high storage capacities, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z, deduplication, compression, multi level caching (with SSD if you like) and encryption.

That time Linux and ZFS wasn't a good company because of licensing issues (it still exists), but ZFS was available through FUSE (Filesystem In User Space). So I repartitioned my drive at home and started using it. That time only for some not so critical data of mine. I did some benchmarking on it too. It's speed wasn't something big because of FUSE but it was OK for me.

Time went by and a company namely Knowledge Quest Infotech started to write a port os ZFS based on Lawrence Livermore National Laboratory's work can be found at http://zfsonlinux.org/. And after several months it became ready for use (in january 2011). The licensing issues are still here so it is only available as module and maybe never will go to the native kernel. Building the modules or download from KQ Stor's (the brand what Knowledge Quest Infotech made for ZFS) is acceptable for me. (Oh sometimes I hate living on the edge... but this is the way I do in informatics.:))

I had several initial issues about the new native ZFS, for example that importing the old pool, what I made with ZFS-FUSE made all my files access mode to 000, but upgrading the file system version resolved this.

The most annoying issue I've found was the memory consumption of ZFS, first I suspected it as a memory leak, but it turned out to be a feature, but not on my desktop. ZFS itself takes very large amount of memory for ARC caching and if it needs by applications it releases them, the problem was that my PC was overloaded, Firefox, Thunderbird, several JAVA apps, etc... so my 3G RAM was almost full, but when ZFS started to allocate many memory and my desktop would need it my computer started to swap despite ZFS started to free memory (it was slower than kswapd). So many times this rendered my PC unusable for minutes. But after discussing it on the lists I learned how to limit ZFS's memory allocation so now everything is perfect. Don't blame ZFS, it's memory allocation is perfect for servers with more than 4G memory but racing with my desktop programs it wasn't a success story. But now it's good and I am happy with it. I've just bought another two 1T winchester to boost my storage space and with ZFS it was faster than I wrote this sentence!

It is still a new comer in Linux but actively developed so these first glitches will be surely resolved in the near future and it will become the most powerful file system in Linux.

It this post awakened your interesting please give it a try.:)

Android 2.2.1 on my Galaxy Spica

Thanks for the guys at Samdroid I am using 2.2 on my phone for a week now and happy with it. Brain Fuck Scheduler is great!:) My phone is faster then ever!

Published with Blogger-droid v1.6.5

Slicehost is my new VPS provider

I've switched to http://slicehost.com because I was a bit disapppinted with my old VPS provider.

Slicehost is an affordable and as far as I use a reliable VPS solution. It brings the newest Ubuntu distros with the possibility of using your own kernel.

The administration interface is great, it gives you console access. The servers aren't overloaded. It gives you cheap backup function too.

Give it a try if you looking for a VPS server!

Published with Blogger-droid v1.6.5

Installing VMware Server on Ubuntu 10.10

Since VMware Server's developement doesn't follow Linux kernel changes, installing on recent distros isn't easy. Fortunately there are great ones who do patches and howto's to help.

I have folloverd the following post http://risesecurity.org/2010/04/02/vmware-server-2-0-2-update-patch-2/ and succesfully installed VMware Server 2.0.2 on my Ubuntu 10.10 box. Thanks!