Archivo de la etiqueta: Linux

task blocked for more than 120 seconds


When we run very stressful jobs on clustered environments, where IO activity is very high. It is pretty common to start seeing these messages on the ‘dmesg’ kernel output: [24169.372862] INFO: task kswapd1:1140 blocked for more than 120 seconds. [24169.375623] … Seguir leyendo

Publicado en Uncategorized | Etiquetado , , | Deja un comentario

failed to alloc buffer for rx queue


If we put enough pressure over the ENA network driver, we’ll start seeing these “failed to alloc buffer for rx queue” messages on the ‘dmesg‘ output. [56459.833033] ena 0000:00:05.0 eth0: failed to alloc buffer for rx queue 4 [56459.836477] ena … Seguir leyendo

Publicado en Uncategorized | Etiquetado , , | Deja un comentario

Listen on a port/Send data to a port


Listening with nc: nc -l 8089 Checking that 8089 port is listening: netstat -nap|grep 8089 (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 … Seguir leyendo

Publicado en Uncategorized | Etiquetado , | Deja un comentario

Move a Linux running process to a screen shell session


Use case: You just started a process (i.e. compile, copy, etc). You noticed it will take much longer than expected to finish. You cannot abort or risk the process to get aborted due to the current shell session finishing. It … Seguir leyendo

Publicado en Uncategorized | Etiquetado , | 1 Comentario

Getting latest EMR release label


Usually latest release label gets updated on EMR’s Whats New page. So a way to getting the last EMR release label would be:   curl -s https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html |grep “(Latest)”|head -n1|awk ‘{ print $3 }’   Have fun!    

Publicado en Uncategorized | Etiquetado , , | Deja un comentario

Update protobuf to 2.5 on Centos 6


Update protobuf to 2.5 on Centos 6: $ cd /usr/local/src/ $ wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz $ tar xvf protobuf-2.5.0.tar.gz $ cd protobuf-2.5.0 $ ./autogen.sh $ ./configure –prefix=/usr $ make $ make install $ protoc –version Install protobuf for java $ cd java … Seguir leyendo

Publicado en Uncategorized | Etiquetado , | Deja un comentario

upgrading glibc from version 2.12 to 2.14 on CentOS


You cannot update glibc on Centos 6 safely, but, you can install 2.14 alongside 2.12 easily, then use it to compile projects etc. mkdir ~/glibc_install; cd ~/glibc_install wget http://ftp.gnu.org/gnu/glibc/glibc-2.14.tar.gz tar zxvf glibc-2.14.tar.gz cd glibc-2.14 mkdir build cd build ../configure –prefix=/opt/glibc-2.14 … Seguir leyendo

Publicado en Uncategorized | Etiquetado , | Deja un comentario

re-indexing Outlook Spotlight index on Mac


If you search on Outlook and you don’t get results or you get partial results, most probably the Spotlight index is corrupted. How to fix it: Restart the Mac, so that it restarts the Spotlight services. Navigate to Finder > … Seguir leyendo

Publicado en Uncategorized | Etiquetado , , | Deja un comentario

Adding a mount point to HDFS


Before proceeding: This procedure considers that you don’t have any current useful data on HDFS. All the data will be lost after adding mount points with this method. This procedure should be applied to every datanode in the cluster. No … Seguir leyendo

Publicado en Uncategorized | Etiquetado , , , | Deja un comentario

AWS EMR – Big Data in Strata New York


Will you be in New York next week (Sept 25th – Sept 28th)?                    Come meet the AWS Big Data team at Strata Data Conference, where we’ll be happy to answer your questions, hear about your requirements, and help you … Seguir leyendo

Publicado en Uncategorized | Etiquetado , , , , , | Deja un comentario