Raspberry Pi + Java R2D2 Robot – Part 1


This project was born as a Coronavirus project for my kids (you know how difficult could be to have your kids all day at home… oh school: please come back, we forgive you!)

On this post we cover the initial steps, including the full platform/chassis running, commanded by a Raspberry Pi and some Java code.

On this video we can see the chassis running:

 

The body of the R2D2 will be a 12″ cardboard tube from Home Depot. The chassis aluminum flat bars are also from HD:

2020-04-18 10.16.05

 

R2D2 head will be molded from a balloon, newspaper paper and paste (flour + water):

2020-04-18 11.46.58

 

Chassis design:

2020-04-18 11.47.28

 

Cutting, drilling and assembling:

2020-04-18 12.28.45

2020-04-18 12.34.452020-04-18 12.53.332020-04-18 12.57.15

 

 

 

 

 

 

 

 

 

 

 

This is how the chassis will fit into the tube:

2020-04-18 12.46.53

 

Soldering:

2020-04-18 13.09.55-4

 

Wheels in place:

2020-04-18 14.02.44

 

Preliminary setup and code testing:

2020-04-18 14.13.242020-04-19 11.05.302020-04-19 12.09.46-22020-04-19 12.09.46

 

 

 

 

 

 

 

 

 

 

Final assembly of the chassis:

2020-04-19 12.42.58

 

We are using two separate battery packs: one for the raspberry and a second one (more amps and 12V) for the motors and the L298n motor driver:2020-04-19 13.02.19

 

To be continued…..

2020-04-19 18.24.02

Publicado en Uncategorized | Etiquetado , | Deja un comentario

Proyecto Raspberry Pi + Java R2D2 Robot – Parte 1


Este proyecto nacio a causa del Coronavirus como una manera de mantener entretenidos a mis hijos (me imagino que ya saben que complicado puede ser tener a los chicos todo el dia en casa… Escuela volve!!)

En este post les muestro los pasos iniciales, incluyendo la construccion de la plataforma/chasis, y su funcionamiento comandado por un Raspberry Pi y algo de codigo en Java

En este video, vemos como funciona hasta ahora con el chassis ya ensamblado:

 

El “cuerp” del R2D2 es un tubo de carton de 12″ de los que se usan en construccion. El chasis esta construido con barras planas de aluminio:

2020-04-18 10.16.05

 

La cabeza del R2D2 se moldea con un globo, papel de diario y engrudo (agua y harina):

2020-04-18 11.46.58

 

Diseño del chasis:

2020-04-18 11.47.28

 

Cortando, taladrando y ensamblando:

2020-04-18 12.28.45

2020-04-18 12.34.452020-04-18 12.53.332020-04-18 12.57.15

 

 

 

 

 

 

 

 

 

 

 

Asi quedara el chasis en el tubo:

2020-04-18 12.46.53

 

Soldando:

2020-04-18 13.09.55-4

 

Ruedas en su lugar:

2020-04-18 14.02.44

 

Armado preliminar y prueba de codigo:

2020-04-18 14.13.242020-04-19 11.05.302020-04-19 12.09.46-22020-04-19 12.09.46

 

 

 

 

 

 

 

 

 

 

Ensambrado final del chasis:

2020-04-19 12.42.58

 

Usamos dos paquetes de baterias recargables: una para el Raspberry y otra (con mas amperaje y de 12V) para los motores y el driver L298N:

2020-04-19 13.02.19

 

Continuara…..

2020-04-19 18.24.02

Publicado en Uncategorized | Etiquetado , | Deja un comentario

task blocked for more than 120 seconds


When we run very stressful jobs on clustered environments, where IO activity is very high. It is pretty common to start seeing these messages on the ‘dmesg’ kernel output:

[24169.372862] INFO: task kswapd1:1140 blocked for more than 120 seconds.
[24169.375623] Tainted: G E 4.9.51-10.52.amzn1.x86_64 #1
[24169.378445] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[24169.382533] kswapd1 D 0 1140 2 0x00000000
[24169.385066] ffff8811605c5a00 0000000000000000 ffff8823645dc900 ffff882362844900
[24169.389208] ffff882371357c00 ffffc9001a13ba08 ffffffff8153896c ffffc9001a13ba18
[24169.393329] ffff881163ac92d8 ffff88115e87f400 ffff882362844900 ffff88115e87f46c
[24169.445313] Call Trace:
[24169.446981] [<ffffffff8153896c>] ? __schedule+0x23c/0x680
[24169.449454] [<ffffffff81538de6>] schedule+0x36/0x80
[24169.451790] [<ffffffff8153907e>] schedule_preempt_disabled+0xe/0x10
[24169.454509] [<ffffffff8153a8d5>] __mutex_lock_slowpath+0x95/0x110
[24169.457156] [<ffffffff8153a967>] mutex_lock+0x17/0x27
[24169.459546] [<ffffffffa0966d00>] xfs_reclaim_inodes_ag+0x2b0/0x330 [xfs]
[24169.462407] [<ffffffffa0967be3>] xfs_reclaim_inodes_nr+0x33/0x40 [xfs]
[24169.465203] [<ffffffffa0978f59>] xfs_fs_free_cached_objects+0x19/0x20 [xfs]
...

This indicates that the process requested a block device such as a disk, and wasn’t able to be fulfilled for more than 120 seconds and subsequently abandoned by the kernel.

The probability of observing this behavior will increase when we use instances with large number of vCores (e.g. 64+ vCores), given that the volume of IO requests could be higher.

Even though the framework will retry, on Hadoop/YARN environments, this will impact in performance, and, might also lead to application failures.

To solve this problem, we have to increase the ‘dirty_background_bytes‘ kernel setting  to higher values to be able to accommodate the throughput.

As a base formula, we usually consider a value of dirty_background_bytes=10MB for 40MB/Sec throughput.

The goal is to have the page cache to allow the OS to write asynchronously to disk whenever possible. Otherwise, when the write becomes synchronous, it would involve a waiting time.

Additionally to the dirty_background_bytes kernel parameter, we can also set:

dirty_background_ratio = 10 (represents the percentage of system memory which when dirty then system can start writing data to the disks. When the dirty pages exceed ratio 10, I/O starts, i.e they start getting flushed / written to the disk)
dirty_ratio = 15 (is percentage of system memory which when dirty, the process doing writes would block and write out dirty pages to the disks. When total number of dirty pages exceed ratio 15, all writes get blocked until some of the dirty pages get written to disk. This is like an upper limit)

We try to keep these two values low to avoid  I/O bottlenecks. We can experiment setting both down to zero, to force fast flushes on high stress environments.

Another parameter we can use is dirty_bytes, which represents the amount of dirty memory at which a process generating disk writes will itself start writeback. We should set dirty_ratio = 0 if we set dirty_bytes, as only one of these might be specified at a time.

e.g.

dirty_bytes = 2147483648

More details on these parameters is available here.

 

 

Publicado en Uncategorized | Etiquetado , , | Deja un comentario

failed to alloc buffer for rx queue


If we put enough pressure over the ENA network driver, we’ll start seeing these “failed to alloc buffer for rx queue” messages on the ‘dmesg‘ output.

[56459.833033] ena 0000:00:05.0 eth0: failed to alloc buffer for rx queue 4
[56459.836477] ena 0000:00:05.0 eth0: refilled rx qid 4 with only 85 buffers (from 168)

This message will raise when the napi handler fails to refill new Rx descriptors, typically due to lack of memory. This situation might lead to performance decrease, given that some requests would have to be rescheduled. 

The code handling these situations can be found here.

The solution for this is related to a memory increase on “min_free_kbytes” kernel parameter. As example:

vm.min_free_kbytes = 11048576

Place the following commands in /etc/sysctl.conf. And load the new setting with:

sysctl -p

It is recommended to have at least 512MB, with a minimum of 128MB for constrained environment. On large instance types running stress jobs (e.g. 64+ vCores + 256GiB + RAM), this value can typically be set to 10MB.

 

 

 

 

Publicado en Uncategorized | Etiquetado , , | Deja un comentario

Count and Say


/*Lets write an algorithm that, given an initial value, will produce the next 'count and say' sequence:
1, 11, 21, 1211, 111221, 312211, 13112221, 1113213211, …
*/

public class CountAndSay{

  public static void main(String[] args){

     System.out.println("Count and Say algorithm");

     System.out.println("next sequence is: " + countAndSay(args[0].toString()));

  }

  public static String countAndSay(String input){

     Boolean change=false;
     String output="";
     int count=1;

     char[] str = input.toCharArray();

     System.out.println("Input is: " + input);

     for (int i=0;i<str.length;i++){

       if (i+1<str.length && str[i]==str[i+1]){
          count++;
       }
       else{
          output = output + count + str[i];
          count=1;
       }

     }

     return output;

  }

}

 

 

Publicado en Uncategorized | Etiquetado , | Deja un comentario

Listen on a port/Send data to a port


Listening with nc:

nc -l 8089

Checking that 8089 port is listening:

netstat -nap|grep 8089
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 0.0.0.0:8089 0.0.0.0:* LISTEN 27543/nc
tcp 0 0 127.0.0.1:8089 127.0.0.1:43592 TIME_WAIT -

Sending data to 8089 port:

telnet localhost 8089
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
hello

 

listeningOnAPort_SendingDataToAPort

 

Publicado en Uncategorized | Etiquetado , | Deja un comentario

Move a Linux running process to a screen shell session


Use case:

  • You just started a process (i.e. compile, copy, etc).
  • You noticed it will take much longer than expected to finish.
  • You cannot abort or risk the process to get aborted due to the current shell session finishing.
  • It would be ideal to have this process on ‘screen’ to have it running on backgroud.

We can move it to a screen session with the following steps:

  1. Suspend the process
    1. press Ctrl+Z
  2. Resume the process in the background
    1. bg
  3. Disown the process
    1. disown %1
  4. Launch a screen session
    1. screen
  5. Find the PID of the process
    1. pgrep myappname
  6. Use reptyr to take over the process
    1. reptyr 1234

 

Note: at the moment of writing this, reptyr is not available on any Fedore/Redhat repo. We’ll need to compile:

$ git clone https://github.com/nelhage/reptyr.git
$ cd reptyr/
$ make
$ sudo make install

 

 

 

 

Publicado en Uncategorized | Etiquetado , | 1 Comentario