Libre Things

marcmaurice.fr Diaspora
fr

Saturday 16 March 2013

The ultimate live USB MultiBoot solution: Easy2Boot

I've spent days looking for a good live Multiboot USB solution. I mean a way to include many downloaded ISO files in a single USB stick. I was looking for a system allowing to drop the ISOs in a folder on the key, just showing a menu with the ISO files to boot. Easy2Boot does exactly that!

Booting an ISO file is complex because lots of live operating systems expect to be at the root of a DOS partition. After some research I found several posts saying that generic ISO chainloading is very hard (p1 p2).

I found solutions like YUMI (Your Universal Multiboot Installer (windows) or MultiSystem. At this time, the problem of those programs is that they use specific "tricks" for each distribution to build a menu that will load directly the Linux kernel. It's not as simple as dropping your ISOs in a folder: you have to use the software to build the multiboot usb key. Another drawback is that you will loose the SYSLINUX boot menus of the distros, because it loads directly the Linux kernels.

You also have hardware usb disk drives emulating a cdrom device. It's probably the most reliable solution, but it's not cheap, and bigger that a small USB key.

Easy2Boot, with grub4dos scripting, uses a nice trick (described here): when the iso file is chosen from the boot menu, a DOS partition (number 4) is created starting and ending at the position of the .iso file on the FAT filesystem. This method is more "ISO generic" and can boot many ISO files (99% of Live Linuxes). The drawbacks is that you have a partition overlapping the other (this is not a problem, at least under Linux), and you have to make sure that the FAT filesystem will not fragment your ISO files when you copy them on the key.

Under Windows, you can use the RMPrepUSB software (freeware/not opensource) to prepare your USB key.

Under Linux/Ubuntu, this is simple too:

  1. Format your USB key with a single FAT partition (I recommend gparted).
  2. Download Easy2Boot (v1 Beta06 works fine) and put all the files at the root of the partition.
  3. Download grub4dos (take the last Featured version) and extract the archive on your PC (not on the usb stick)
  4. From the grub4dos folder, run sudo ./bootlace.com --time-out=0 /dev/sdX to install grub4dos MBR (make sure you use the right device!).
  5. Put some ISOs directly in the /MAINMENU directory
  6. Here you go. Reboot and enjoy!

Thanks to RMPrepUSB and to the reboot.pro forum guys!

Hints:

Links:

Saturday 8 September 2012

Listen to mouse events in Linux (text mode)

A small script to listen to mouse events with the mev command using gpm (General Purpose Mouse).

Can be used for example on the Raspberry Pi to run mpd commands, to use a mouse as a remote control.

I didn't find how to isolate the wheel events in order to control the volume. Any idea somebody?

In Archlinux, you can add the following command in /etc/rc.local to start it at boot time:

nohup /usr/local/bin/mpd_mouse.sh > /tmp/mpd_mouse.log 2>&1 &
#!/bin/sh
# This script listen to mouse events with the mev command using gpm.
# Can be used for example on the Raspberry Pi to run mpd commands, to use a mouse as a remote control.
# You can start it as a daemon with:
# nohup /usr/local/bin/mpd_mouse.sh > /tmp/mpd_mouse.log 2>&1 &

# start gpm if not already started
gpm -m /dev/input/mice -t imps2

# unset TERM variable, otherwise mev refuses to start when detecting xterm
unset TERM

echo "Listening to mouse events..."

# we use script to fake a tty for mev, otherwise it exits (note: mev logs errors in syslog)
script -qc "mev -E" /dev/null </dev/null | grep --line-buffered -v "mouse-movement" | while read LINE
do
        echo
        echo "$LINE"

        EVENT=$(echo "$LINE" | cut -d' ' -f1 | cut -d'(' -f2)

        if [ "$EVENT" = "down-mouse-1" ]
        then
                echo mpc stop
                mpc stop
        elif [ "$EVENT" = "down-mouse-2" ]
        then
                echo mpc toggle
                mpc toggle
        elif [ "$EVENT" = "down-mouse-3" ]
        then
                echo mpc next
                mpc next
        else
                echo "nothing"
        fi

done

ssh-agent: automatic ssh-add on demand

Several desktop environments (Gnome, KDE) automatically start an SSH agent at startup. However, you have to think of running ssh-add before connecting to a server.

Waiting for automatic ssh-add in OpenSSH, you can add this to your .bashrc:

ssh-add -l >/dev/null || alias ssh='ssh-add -l >/dev/null || ssh-add && unalias ssh; ssh'

The alias is created only if the identity is not added, and the alias destroys itself once run.

This way the regular ssh command is used after the identity has been added.

http://superuser.com/questions/325662/how-to-make-ssh-agent-automatically-add-the-key-on-demand/471640#471640

Friday 11 May 2012

An outgoing port tester

The server hosting this blog respond on all ipv4 TCP ports from 1 to 65535.

See: http://portquiz.positon.org

The following URLs should also work if you are not filtered :

This allows you to test what TCP ports are filtered by your local network.

Command line example with netcat:

$ nc -v portquiz.positon.org 1
electron.positon.org [178.33.250.62] 1 (tcpmux) open
$ nc -v portquiz.positon.org 65535
electron.positon.org [178.33.250.62] 65535 (?) open

Sunday 29 April 2012

Pixel crisis: resolution regression

I'm looking for a new laptop. My old T42p Thinkpad is becoming old. I'm looking for a laptop with a good screen...

However... finding a laptop with a resolution similar to my T42p is impossible on today's market. The T42p has been released for more than 8 years now!! When I'm speaking resolution, I mean pixel density in PPI. My resolution is of 1400x1050 for 14.1 inches, which is good! it gives a density of 124 PPI.

Nowadays, all laptops have the same unique 1366x768 resolution. The last thinkpad X1 from Lenovo with 13.3 inches is only 118 PPI.

It's revolting! I want my pixels!

Apparently I'm not the only one in this situation...

It's sad. Maybe it will be better in the future... with our cell phones screens having far better resolutions...

Other usefull links :

Monday 16 April 2012

Compare / diff between two images

After searching a long time on the Net, here is how to use ImageMagick to compare two images (diff), to determine if the images are similar, or if one image is a resize version of the other.

convert image1 image2 -resize '400x300!' MIFF:- | compare -metric AE -fuzz '10%' - null:

The convert command takes 2 images and scale them to a smaller identical size. We then pipe them to the compare command. The compare command will count the number of different pixels.

The command displays the number of different pixels. If it's zero then pictures are similar.

2 parameters can increase the similarity tolerance:

  • The size of thumbnails to compare: the smaller it is, the more details are removed. You can use 1/4 of the smaller image for example.
  • The -fuzz parameter: the color distance tolerance. The more you increase this param, the more you allow different colors. Color difference is almost undetectable below 2%.

Notes:

  • It's better to keep ratio while generating thumbnails.
  • The exclamation mark is needed to force image scale without preserving ratio. Otherwise in some cases the 2 thumbs are not strictly the same size and the compare command fails.

I also made a small script to scale at 1/4 of the small image and displays the percentage of different pixels: imdiff

./imdiff /tmp/bad.jpg ../Public/images/bad.jpg
pixel difference: 2.927%
NOK

Links:

Saturday 25 February 2012

Android HTC Dream AZERTY keyboard

Small contribution to CyanogenMod wiki

Physical keyboard is changed to QWERTY after CyanogenMod 6 flash. if your HTC Dream has an AZERTY keyboard:

adb remount
adb shell
cat /sdcard/trout-keypad-v3.kl > /system/usr/keylayout/trout-keypad-v3.kl
cat /sdcard/azerty.kcm.bin > /system/usr/keychars/trout-keypad-v3.kcm.bin
  • Then reboot your phone

Some doc: http://source.android.com/tech/input/key-layout-files.html

Monday 30 January 2012

Remove Mobipocket DRM (.prc file)

The user manual of my Bookeen Cybook Gen 3 is DRM protected ! irrk..

Download last version of DRM Tools Archive here : http://stream-recorder.com/forum/download-drm-tools-archive-t7955p3.html

Extract the archive, then :

tools_v4.7/Mobi_Additional_Tools$ python MobiDeDRM_032.py /tmp/XXXXX.prc /tmp/decrypted.prc

You can now open the file with fbreader.

I'm not sure it works with all PRC files. Maybo you have to give the device PID for some.

Sunday 13 November 2011

Import CSV file to MySQL

Here is the little sh script I made to do that. The LOAD DATA INFILE command exists but is not capable of creating the table structure.

#!/bin/sh

MYSQL_ARGS="--defaults-file=/etc/mysql/debian.cnf"
DB="mbctest"
DELIM=";"

CSV="$1"
TABLE="$2"

[ "$CSV" = "" -o "$TABLE" = "" ] && echo "Syntax: $0 csvfile tablename" && exit 1

FIELDS=$(head -1 "$CSV" | sed -e 's/'$DELIM'/` varchar(255),\n`/g' -e 's/\r//g')
FIELDS='`'"$FIELDS"'` varchar(255)'

#echo "$FIELDS" && exit

mysql $MYSQL_ARGS $DB -e "
DROP TABLE IF EXISTS $TABLE;
CREATE TABLE $TABLE ($FIELDS);

LOAD DATA INFILE '$(pwd)/$CSV' INTO TABLE $TABLE
FIELDS TERMINATED BY '$DELIM'
IGNORE 1 LINES
;
"

http://dev.mysql.com/doc/refman/5.1/en/load-data.html

(See comment: "Posted by John Swapceinski on September 5 2011 5:33am")

Tuesday 14 June 2011

PHP escapeshellarg function, UTF8 and locales

The PHP escapeshellarg function depends on your current locale. I think it's bad, but PHP developers made this choice. If like me your default locale is 'C' you lose all UTF8 characters.

They suggest you to call something like setlocale(LC_CTYPE, "en_US.UTF-8"). It doesn't work if the en_US.utf8 locale is not installed on your system. Of course maybe you have the fr_FR.utf8, or de_DE.utf8, but you will have to try all of them until you find one utf8 matching locale. And if there is not, you're screwed. It's also bad if you want code that runs everywhere.

Simply use that:

$escapedArg = "'".str_replace("'", "'\\''", $arg)."'";

It will do the same as the escapeshellarg function: replace yourstringthat'slong by

'yourstringthat'\''slong'

as described in the escapeshellarg manual (and I also looked into the PHP source code to be sure).

See also:

Sunday 13 March 2011

A very simple mysqldump script to backup your databases

Here are some lines I'm using to backup my MySQL databases on my Debian server:

#!/bin/sh
# This will dump all your databases

DATE=$(date +%Y%m%d%H%M)

for DB in $(echo "show databases" | mysql --defaults-file=/etc/mysql/debian.cnf -N)
do
        mysqldump --defaults-file=/etc/mysql/debian.cnf $DB > /backup/mysql/${DB}_${DATE}.sql

        gzip /backup/mysql/${DB}_${DATE}.sql
done

# purge old dumps
find /backup/mysql/ -name "*.sql*" -mtime +8 -exec rm -vf {} \;

You can run it in a cron:

11 1 * * * /usr/local/bin/mysqldump.sh > /tmp/mysqldump.log

This way any error displayed by the script will be sent by mail to the root user (mail address in /etc/aliases).

If you are not under Debian and there is no password file in /etc/mysql, you should create such file.

Thursday 13 January 2011

positon.42 and dnsmasq as a DNS proxy to resolve .42 domains

I am now the owner of a .42 domain !

http://positon.42

.42 are not official domains (yet). Official top level domains are managed by ICANN and served by root DNS servers.

To resolve .42 domains, you have to query a DNS server knowing .42 domains.

Here is my small contribution to the 42registry.org wiki, explaining a way to configure it with Ubuntu:

---

The following allows you to use Geeknode DNS only for .42 domains while keeping your regular DNS provider for other domains.

Install dnsmasq :

aptitude install dnsmasq

Edit /etc/dnsmasq.conf and add the line :

server=/42/81.93.248.69

Restart dnsmasq :

sudo /etc/init.d/dnsmasq restart

Edit /etc/dhcp3/dhclient.conf, then uncomment or add the line :

prepend domain-name-servers 127.0.0.1;

Now disconnect and reconnect to you local network, to refresh the /etc/resolv.conf file, then test it !

---

Links:

Wednesday 29 December 2010

Rsync command restriction over SSH

You have 2 systems and you want to set up a secure backup with rsync + SSH of one system to the other.

Very simply, you can use:

backup.example.com# rsync -avz --numeric-ids --delete root@myserver.example.com:/path/ /backup/myserver/

To do the backup, you have to be root on the remote server, because some files are only root readable.

Problem: you will allow backup.example.com to do anything on myserver.example.com, where just read only access on the directory is sufficient.

To solve it, you can use the command="" directive in the authorized_keys file to filter the command.

To find this command, start rsync adding the -e'ssh -v' option:

rsync -avz -e'ssh -v' --numeric-ids --delete root@myserver.example.com:/path/ /backup/myserver/ 2>&1 | grep "Sending command"

You get a result like:

debug1: Sending command: rsync --server --sender -vlogDtprze.iLsf --numeric-ids . /path/

Now, just add the command before the key in /root/.ssh/authorized_keys:

command="rsync --server --sender -vlogDtprze.iLsf --numeric-ids . /path/" ssh-rsa AAAAB3NzaC1in2EAAAABIwAAABio......

And for even more security, you can add an IP filter, and other options:

from="backup.example.com",command="rsync --server --sender -vlogDtprze.iLsf --numeric-ids . /path/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAAAB3NzaC1in2EAAAABIwAAABio......

Now try to open a ssh shell on the remote server.. and try some unauthorized rsync commands...

Notes:

  • Beware that if you change rsync command options, change also the authorized_keys file.
  • No need for complex chroot anymore. Forget my previous article: /post/SFTP-chroot-rsync

See also:

  • man ssh #/AUTHORIZED_KEYS FILE FORMAT
  • man rsync
  • view /usr/share/doc/rsync/scripts/rrsync.gz (restricted rsync, allows you to manage allowed options precisely)

Sunday 19 December 2010

I2P proxy configuration via PAC file

With the new french Hadopi law... I just tested the I2P nework, which works fairly well.

Installation is very easy. Just download the .exe file. Then launch it with the java -jar command, like explained on the official site (and it works on Linux, yes !).

Now, if you want to access .i2p sites, like http://forum.i2p, you must configure your browser to use the I2P proxy: localhost:4444

If you don't want to use the proxy when connecting to standard non-i2p sites, you can use the FoxyProxy Firefox extension.

You can also use a PAC (Proxy Auto-Config) file:

function FindProxyForURL(url, host) {
    if (dnsDomainIs(host, ".i2p")) {
        return "PROXY localhost:4444";
    } else {
        return "DIRECT";
    }
}

Then configure Firefox, or proxy configuration in Gnome to use the file:///path/to/proxy.pac file as configuration.

Warning: a bad eepsite (.i2p site), can detect your real IP address with that.

Links:

Tuesday 30 November 2010

Clementine: a good music player

I discovered Clementine some time ago.

There is not any package for Ubuntu or Debian, but the official site gives you a .deb for your favorite Ubuntu version. And you'll also find Mac OS X and Windows versions...

A small click on the good .deb file for your Ubuntu distro and it should install.

I tested tons of players (Amarok, Rythmbox, old XMMS, Exaile, Listen, Totem...).

Clementine is based on Amarok, it's far simpler and integrates very well in Gnome desktop using Qt4 librairies.

I'm happy to know this software and I recommend it to you.

Tuesday 9 November 2010

Exim Router: recipient filter

In a previous post, I explained how to setup a catchall with Exim. This to create multiple addresses all redirecting to the same destination:

catchall_pub:
  driver = redirect
  domains = ads.mydomain.com
  data = user@mydomain.com

Then, I wrote that on destination address we could setup a Sieve or Exim filter with a .forward file.

If you want to do that directly in Exim, to have all config in one place, or simply if your destination mail box doesn't supports filters...

Just add a local_parts option:

catchall_pub:
  driver = redirect
  domains = ads.mydomain.com
  local_parts = !/etc/exim4/ads.mydomain.com.blacklist
  data = user@mydomain.com

In /etc/exim4/ads.mydomain.com.blacklist we put all rejected addresses, one per line:

foo1
bar2

This way foo1@ads.mydomain.com and bar2@ads.mydomain.com will be rejected.

Exim4 Doc:

Saturday 30 October 2010

Preload web content using link prefetch or javascript

For the Bizou php gallery, I looked for different ways of preloading next image in "view" mode (example).

With Firefox it's very simple. Just use the following element and the browser will preload your contents. Contents are preloaded in background, once the whole current page is loaded.

<link rel="prefetch" href="/images/nextimage.jpg" />

Problem: only Firefox supports this currently. Note: a ticket is opened about this in the Chromium project.

For other browsers, use some Javascript triggered by the window.onload event:

<script type="text/javascript">
window.onload = function() {
    // for images
    var im = new Image();
    im.src = '/images/nextimage.jpg';
    // and for other content
    var req = new XMLHttpRequest();
    req.open('GET', 'nextpage.php', false);
    req.send(null);
};
</script>

Beware of HTTP cache headers sent by the server to the browser. To preload correctly PHP pages, make your script send an Expires header:

header('Expires: '.gmdate('D, d M Y H:i:s \G\M\T', time() + 3600));

Then, for a simple browser detection from your PHP script:

<?php if (strpos($_SERVER['HTTP_USER_AGENT'], 'Firefox') !== false) { ?>
<link rel="prefetch" href="nextpage.php" />
 
<?php } else { ?>
<script type="text/javascript">
window.onload = function() {
    var req = new XMLHttpRequest();
    req.open('GET', 'nextpage.php', false);
    req.send(null);
};
</script>
<?php } ?>

Links :

Sunday 24 October 2010

Bizou - a (french) KISS php image gallery

I spent hours on the Web looking for THE PHP software I need with no success.

Required features:

  • Free-libre software
  • No database, just images in a directory, that's all.
  • Easy to understand source code, easy to patch for my needs
  • Variable number of thumbs fitting the browser width.

And I spent another weekend happy coding something that may already exist. But my gallery is simple and does what I want:

http://www.positon.org/bizou/en.html

Demo here

Friday 22 October 2010

A solution to the umask problem: inotify to force permissions

Finding a good solution for sharing files between Linux users is a nightmare.

If using a unique UID is not a problem, it's the most simple solution. All clients access files with the same UID. This way you cannot know who does what, and users cannot fine tune access rights.

The problem: default umask is ALWAYS 0022, so that any created file will get rw– r–– r–– permissions. Only the owner can write. Nobody else. To share files, a group must have write access.

You can change the umask. For command line, you set it in .bashrc or .profile, or /etc/profile for all users. For a SFTP share, you can set it with a trick. For Apache HTTP server, you can set it with /etc/apache2/envvars under Debian.

If file sharing is only done via on service, changing umask is simple, otherwise it's not that easy. And even if you change umask for all services, nothing is perfect: for example it doesn't work with Nautilus and SFTP. Some clients drop files and issue a chmod right after: the hell. You can also try the power of POSIX ACL to force permissions. But problems still remain with some clients.

And for the umask, maybe you don't want all files to be dropped group writable. Maybe you want more granularity on permissions.

So I abandonned the idea of fixing the problem at the source in favor of some trick AFTER file creation. The most simple solution is the cron task: every X minutes, run chmod -R g+w on the directory. This way permissions are not fixed immediately, but asynchronously. And it adds a (very) little more load to your system.

My solution uses inotify to listen for file changes and force permissions when files are created:

aptitude install inotify-tools

And the magical command:

inotifywait -mrq -e CREATE --format %w%f /tmp/mytest/ | while read FILE; do chmod g=u "$FILE"; done

UPDATE 2010-10-30 To support spaces at the end of filenames, and backslashes, use:

inotifywait -mrq -e CREATE --format %w%f /tmp/mytest/ | while IFS= read -r FILE; do chmod g=u "$FILE"; done

Thanks to vitoreiji (see comments)

inotifywait listens for events in the /tmp/mytest directory. When a file is created, it's displayed on standard output. Then each fileline is read by the while loop and permissions are changed. g=u gives the group the user's permissions (with g+w, if the user drops a file with rw– ––– –––, permissions will be rw– –w– –––).

You can now test file/directory creation and copy. mkdir -p a/b/c/d/e shoud also work.

Finally, add it in a boot script:

vi /usr/local/bin/inotifywait.sh && chmod +x /usr/local/bin/inotifywait.sh
#!/bin/sh
# Take the directory name as argument

inotifywait -mrq -e CREATE --format %w%f "$1" | while read FILE
do
	chmod g=u "$FILE"
done
vi /etc/init.d/inotifywait.sh && chmod +x /etc/init.d/inotifywait.sh
#! /bin/sh

case "$1" in
  start|"")

	rm -f /tmp/inotifywait.log
	/usr/local/bin/inotifywait.sh /path/to/dir/ >/tmp/inotifywait.log 2>&1 &
	
	;;
  restart|reload|force-reload)
	echo "Error: argument '$1' not supported" >&2
	exit 3
	;;
  stop)
	# killall inotifywait ???
	;;
  *)
	echo "Usage: inotifywait.sh [start|stop]" >&2
	exit 3
	;;
esac

:

(Debian way)

update-rc.d inotifywait.sh defaults

Note: a drawback: there is a limit on the number of tracked files. See -r option in man inotifywait.

Then the final touch in order for the new files to be created with the same group as their parent: setgid bit for all directories.

find /path/to/dir -type d -exec chmod g+s {} \;

Links:

Monday 11 October 2010

ProFTPD and AuthUserFile for password file

Default in Debian: ProFTPD only uses the /etc/passwd system user base.

To add "virtual" users :

In /etc/proftpd/proftpd.conf:

DefaultRoot ~
AuthUserFile /etc/proftpd/ftpd.passwd
RequireValidShell off

Then restart proftpd: /etc/init.d/proftpd restart

Create users file:

vi /etc/proftpd/ftpd.passwd
username:HASH:1011:1011:MyUserName:/home/ftp/directory:/bin/true

You can also generate the user lines with the ftpasswd command, but I think it's simpler to generate passwords with the command:

mkpasswd --hash=md5

Then paste the HASH in the passwd file.

Correct passwd file permissions:

chown proftpd /etc/proftpd/ftpd.passwd
chmod go-r /etc/proftpd/ftpd.passwd

And just test it to finish.

Links:

- page 1 of 3