Launching Shell Scripts in the Background so Logout Doesn’t Kill them
nohup is the command that stops the task being killed on logout.
nohup name_of_scipt.sh arg1 arg2 … argn > batch.out 2> batch.err < /dev/null &
Breaking it down you have nohup followed by the script and its arguments (if there are any). Next you pipe the various outputs to the files xxx.out, xxx.err and /dev/null/ (a kind of balckhole). Finally and very importantly the ampersand at the end forces the process into the background.
HBase Too many open files, OS X 10.6.8
Today I ran into the following error
2011-12-18 13:02:52,918 FATAL org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Replay of hlog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: .META.,,1 at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:946) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:839) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:241) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:149) Caused by: java.io.FileNotFoundException: /Users/maxgarfinkel/Hadoop/hbase-datastore/hbase-maxgarfinkel/hbase/.META./1028785192/info/223659818943414297 (Too many open files)
This is a well documented error and is caused by HBase opening more files than the OS will let it. To be clear this is an OS configuration issue. On the mac OS the default max number of files a process can open is 264, this