Following in the spirit of Hidden Features of PowerShell and various others on Stack Overflow, what Linux commands or combinations of commands do you find essential in your work?
Also See:
Hidden Features of MySQL
Hidden Features of PowerShell
Hidden features of Oracle Database
Hidden Features of Windows 2008
Hidden Features of Solaris/OpenSolaris
Hidden Features of SQL Server
Hidden Features of IIS (6.0 / 7.0)
-
To get the ball going, I find screen to be essential:
When screen is called, it creates a single window with a shell in it (or the specified command) and then gets out of your way so that you can use the program as you normally would. Then, at any time, you can create new (full-screen) windows with other programs in them (including more shells), kill the current window, view a list of the active windows, turn output logging on and off, copy text between windows, view the scrollback history, switch between windows, etc. All windows run their programs completely independent of each other. Programs continue to run when their window is currently not visible and even when the whole screen session is detached from the users terminal.
xkcd150 : has anyone got anything to say about the screen profiles rolling out with ubuntu 9.04? i looked at them for a bit but didn't give them enough time to be able to say if they're actually useful or just look nicesetatakahashi : OMG! screen again?staticsan : @xkcd150, they're pretty similar, but the biggest win is the status line. It shows all your windows plus a heap of stuff about the machine.From Chris Bunch -
grep, awk and sed
top
jayrdub : Those aren't very hiddenFrom Brent -
It's open source. Nothing is "hidden" if you bother to look.
Spoike : Unfortunately open source tends to make "hidden" things rather difficult for you to look.Tom Wright : I'd have argued the opposite - if a feature is hidden in a closed source app, it's up to the dev shop to disclose it. In an open source app, the feature is there for everyone to see. Furthermore, the number of people actively developing is generally much larger (more people know and talk about 'hidden' features). Lastly, open source development is largely community oriented, so people generally tell each other about hidden features.Paul Tomblin : I'm not following @Spoike's reasoning at all. Yes, there is more to look at, but what you see is what you get.spoulson : I think you're arguing over semantics. How about we rename to "Obfuscated features of linux?"Jason Mock : I would *partially* argue in Spoike's favor that maybe things are 'hidden' because of the general lack of good *up to date* documentation. Not to say this statement holds true for all projects, however there are a great deal of projects where full documentation (and I don't mean just how-to user guides) are somewhat lacking. KDE's administrative features, especially in regards to management of settings inside rc files and actual effects of setting a specific field immutable is a perfect example. In many cases I've found myself looking in the source code to find out how things are handled.From Paul Tomblin -
Network stack can be left running after a system halt. I don't know if this is current with the 2.6.x series of kernels, but on older versions, you could configure the firewall/routing, then halt the system without a shutdown, leaving just the network stack running. This would allow you to make a cheap (although static) firewall that "can't be hacked" - because there's nothing to hack, as there are no programs or services, just the network stack portion of the kernel passing packets back and forth...
wzzrd : Wow. Really cool! +1 I am certainly going to try this out on a 2.6 kernel when I have some spare time.cdeszaq : That's awesome, but how is it unhackable? Doesn't the kernel still have to be running? Or is it a minimal kernel?Avery Payne : "Kernel"? What is this "Kernel" you speak of? When it goes into this mode, the only code running is the /network stack/. No task scheduling, no APIs, code, services, user space, nothing, zip, nada...just the faint sound of packets echoing through ethernet... Think of it as more of an embedded device with two functions in life - accept, and send, packets. I suppose you could "hack" this in the sense that you can try and sneak packets through, but there's nothing you can /directly/ attack in the stack itself.From Avery Payne -
- man (gives help on most commands)
- less (sanely browseable viewing)
- tail -f (view appended data of a file as it grows)
- watch (execute a program and view its output periodically full-screen)
Matthew Flaschen : I find /usr/bin/most even better than less. Excellent syntax highlighting, among other things.niXar : tail -F is better than -f; it notices if a file is rotatedDennis Williamson : @Matthew: I don't get any syntax highlighting in most by itself, nor do I see any reference to it in the docs.Matthew Flaschen : @Dennis, try PAGER=most man lsDennis Williamson : @Matthew: That's not syntax highlighting. That's colorization. You can colorize `less` (including for `man` pages) by setting the `LESS_TERMCAP_*` variables. If `most prog.c` worked, *that* would be syntax highlighting.Matthew Flaschen : Thanks for the correction. Still, you can understand why I don't want to fiddle with termcap variables manually.From Wayne Koorts -
Virtual consoles. Most people know how to use (Ctrl-)Alt-F1 to get to the first console, etc., but what if you have more than 12? You can use Alt-Left and Alt-Right to cycle through the rest of them. :-D
Roy Rico : wow, dude, at that point, just use screen. hehe :) i have it in my .screenrc to show a taskbar at the bottom letting me know screens are openFrom Chris Jester-Young -
I was surprised to find that you can run remote GUI applications over SSH, using the "-X" parameter. For example:
# on my machine $ ssh -X linuxserver # on remote machine $ gedit /etc/my.cnf &
The gedit window appears on my local machine, editing the "my.cnf" file on the server.
I'm assuming this only works if your client machine has an X environment -- in other words, not on Windows. But it works great on my Mac!
xkcd150 : using xming http://sourceforge.net/projects/xming you can have it work on windows, too!gyaresu : SSH forwarding does encryption which slows everything down. If you're on a secure LAN you can just send programs from one Linux desktop to another Linux computers X session. Very handy. I used to run movies via mplayer like that on my main desktop which was plugged into the stero at one end of the lounge room. The mplayer audio would then go out the stereo but the image would got to my laptop (via ethernet) at the other end of the lounge room which was plugged into the data projector. Home movie awesomeness.Zoredache : Instead of adding the -X each time you could update your ssh_config with a host line that matches your internal hosts and address space 'Host *.domain.org 192.168.*.*' and enable the option 'ForwardX11 yes'Notmyfault : I've grown accustomed to -Y, never learned the difference though.wazoox : -Y is trusted X11 forwarding. It's way more secure than -X (simple X11 forwarding). BTW many default sshd setups won't allow -X now, only -Y. If using -X you encounter strange errors, apps closing with "MIT MAGIC COOKIE" errors, then use -Y. Conclusion : don't use -X, only -YFrom Matt Solnit -
Maybe I don't use these every day, but I use them frequently:
- strace Check out what files are loaded by the process.
- htop A nicer top.
- mtr ping + traceroute combined
- lynx/links/w3m In case you need console browsing
- ettercap Great network sniffer (i prefer it over wireshark)
- scripting bash Every *nix admin should know this.
- A programming language. For the more complex things, stay away from bash scripting and use something like python/perl/ruby/tcl/... (I use Lisp)
- Midnight Commander can be great for people who liked norton commander.
- irssi You never know when you just want to go ask something on IRC.
- wget / curl Download stuff from the command line.
- scp Copy stuff over ssh
- lftp / ncftp Good (scriptable) console FTP clients.
- iotop Check what's stressing your disks
- nmap good port scanner
bedwyr : +1 for strace -- absolutely fantasticOddmund : Thanks for the htop and iotop tips!The Pixel Developer : htop is awesome, many thanks.Pier : I would add atop tooFrom Gert M -
I like "locate" - a much easier way to find files than the gnarly "find . -name xxxx -print". Note that you have to use the updatedb command with it to a your file index up to date; see the man pages for details.
Matt Solnit : Luckily, many Linux distros include updatedb in cron.daily :-).altCognito : It's also much, much faster than find.docgnome : It should be pointed out that some people see locate as a security risk. That being said, I use it all the time :-)niXar : Recent verions of locate have the -i switch for ignoring case, very handy.jae : And as to security, there are secure locate versions, which let you "find" only those files the logged-in user can actually see.Mircea Vutcovici : I am keeping disabled updatedb for VMs. They are polluting the file system cache. I prefer that memory to be used by balloon driver or by FS cache for caching normal files rather than the directory entries.ptman : It should be noted that find has a much wider selection of options. find is one of the cornerstones of UNIX and getting familiar with it (and xargs) really pays off.From gareth_bowles -
apropos
DESCRIPTION Each manual page has a short description available within it. apropos searches the descriptions for instances of keyword.
gyaresu@debian:~/bin$ apropos ettercap etter.conf (5) - Ettercap configuration file ettercap (8) - (unknown subject) ettercap_curses (8) - (unknown subject) ettercap_plugins (8) - (unknown subject)
mctylr : I always use`man -k` instead but I'm spelling impaired.From gyaresu -
lsof often gets ignored, its a very useful tool. lsof lets you view a list of every open file on the system, who / what is using it, etc.
For instance:
root@tower:~ # umount /mnt/hardy umount: /mnt/hardy: device is busy umount: /mnt/hardy: device is busy root@tower:~ # lsof | grep /mnt/hardy bash 5966 root cwd DIR 253,2 1024 2 /mnt/hardy root@tower:~ #
Now I see that I'm logged into a shell in another terminal, and /mnt/hardy is my current working directory. So I can either kill that shell, or go to the other terminal and get out of that directory so it can be unmounted.
That's really a trivial example, its very handy for cleaning up the occasional 'bot' infestation too. The options are extensive, see man lsof for more.
bdonlan : fuser is another nice trick, if you're only interested in one file or one mountpoint.altCognito : Hidden for sure. I've been looking for this feature for ages. (I'm a noob though)ptman : lsof is truly wonderful. Combined with the UNIX idea of everything (well, almost) being a file, it can do amazing things. Many things that at first seem impossible or hard to do turn out to be solvable with a oneliner thanks to some of lsof's options.From Tim Post -
I find "ngrep" really useful for debugging network code on remote servers without having to punt tcpdump files around:
ngrep -d any -W byline port 80
for example, will show you live HTTP requests and responses.
One other thing I've found useful frequently is the "-e" switch to strace:
strace -p <pid> -e trace=open
will show all open() syscalls for a given pid, and
strace -p <pid> -e trace=\!rt_sigprocmask
will exclude all calls to rt_sigprocmask() from the output (useful for debugging RoR code, which seems to make an awful lot of those calls when built to use pthreads).
THEn : Now I really starting to love linux. :)From Jon Topper -
Perhaps one of the great features of Linux, as opposed to Windows, is that there are next to no hidden features by design. Hopefully if any undocumented features are important enough to know about the community does its job.
From Andrew H -
Start with this article with some great Linux tricks. My favorite is pstree, which displays your processes in a tree format so you can see which process spawned which.
curl beats the hell out of wget for pretty much anything you would use wget for.
zgrep and zless are great for searching gzipped log files so you don't have to pipe them through gunzip or leave uncompressed stuff around in /var/log.
username : Synopsis, for the lazy ;-) 1.pgrep (like ps meets grep) 2.pstree (lists the processes in a tree format) 3.bc (an arbitrary precision calculator language) 4.split (splits large files into chunks) 5.nl (numbers lines) 6.mkfifo (make two commands communicate through a named pipe) 7.ldd (prints shared library dependencies) 8.col (strips out line feeds) 9.xmlwf (checks if an XML document is well formed) 10.lsof (lists open files)ptman : (GNU) ps has flags for printing a tree. Is there anything other than being easier to remember going on for pstree?From Magus -
Usually people run their desktop system with X. But you can usually access true text-mode terminal, several of them actually. These are called virtual consoles. You can then normally login and use command-line, etc.
Just hit Ctrl + Alt + F1, Ctrl + Alt + F2, etc. Your X-server is generally running at Ctrl + Alt + F7.
This of course is distribution and configuration dependent.
Also this command to kill the X-server is sometimes useful: Ctrl + Alt + Backspace.
These key combinations work at least on i86 PC's.
LiraNuna : ctrl-alt-backspace is disabled on Ubuntu 9.04 (Code name jaunty). Follow this guide if you want to enable it https://wiki.ubuntu.com/X/Config/DontZapFrom Juha Syrjälä -
The concept is that everything in Linux is a file.
All configurations are in text files, and everything in Linux is treated as a file. This is a much simpler approach which makes it very easy to change things in Linux. In Linux even your filesystem itself can be viewed as a file.
wazoox : If you like this basic Unix concept, you'll like Plan9 OS.From ecleel -
NAME units -- conversion program DESCRIPTION The units program converts quantities expressed in various scales to their equivalents in other scales. The units program can only handle multiplicative scale changes. It cannot convert Celsius to Fahrenheit, for example. It works interactively by prompting the user for input: You have: meters You want: feet * 3.2808399 / 0.3048 You have: cm^3 You want: gallons * 0.00026417205 / 3785.4118 You have: meters/s You want: furlongs/fortnight * 6012.8848 / 0.00016630952 You have: 1|2 inch You want: cm * 1.27 / 0.78740157
From gyaresu -
Here are a few I've used on a regular basis:
sar
- shows system activityvmstat
- virtual memory statsiostat
- io statspkill
- likepgrep
but allows you to kill the returned process IDxargs -I<string>
- allows for replacing strings with piped dataat
- schedule a tasktkdiff
- graphicaldiff
utility
These are probably not "hidden", but I find them extremely useful:
df -hk
- show disk usage in human readable formatls -ltr
- list files sorted by datewhile :; do...done
- (Bash) replacement forwatch
if unavailableperl -e
- run aPerl
snippet on the command-linefree -kt
- show memory information (kilobytes w/total)
ptman : I think it's easier to write `while true; do ...` than `while [ 1 ]; do ...`From bedwyr -
A great "feature" I use every day at work: The ability to have SSH listen on port 443 so I can create a tunnel which bypasses my work firewall, allowing me to run a local SOCKS proxy tunneled through SSH to my internet facing Linux server.
I can completely ignore my corporate firewall.
docgnome : It always make me chuckle to think of the people who believe that you can "lock down" a network. If leave a single port for traffic to get out on, you're screwed.Wadih M. : @WerkkreW: what if they see 3GB of data flowing through port 443 from your computer?From WerkkreW -
It's not really that hidden, but may be so for those with no experience, but I really like being able to give lists and let bash expand them like in:
cp arq{,.bak}
which is the same as typing
cp arq arq.bak
I also use the history shortcuts (I don't think that's the right term, but...) like
!!
to repeat the last command, or
^foo^bar
to replace foo by bar in the last command
From Flávio Amieiro -
I find the whereis and which commands handy. Use these when you have alternative versions of the same application with the same name and want to use a specific one of them.
From Dana the Sane -
Thankfully I've only needed a couple of times, but the Magic SysRq key still remains one of my all time favorite hidden features.
Alt+SysRq+RSEIUB
wzzrd : +1 Magic SysRQ is golden. Remember 'Raising Elephants Is So Utterly Boring' as a hint in which order to use the keys, though the order is not set in stone. Which is why RSEIUB is used above: it syncs data to disk before giving the 'terminate' and 'kill' commands instead of the other way around. Correct order is debatable.Coops : +1 Certainly a brilliant one to remember.From gharper -
scp
I'm a web dev and our development setup requires me to push files over to our dev server all the time. I wrote a quick wrapper around scp to handle this for me.
Kevin M : I don't think that scp is a hidden feature; it's more like the standard way of pushing files from box to box without nfs.niXar : I would recommend using sftp instead of scp for scripting. Scp has some weird behaviour with space in filenames.From docgnome -
The hidden feature is that there are no hidden features. The system provides a tremendous amount of power that you have complete access to. You have to understand every piece of it and every tool available to know what power is at your fingertips.
You should start by understanding every command in /bin, /sbin, /usr/bin, /usr/sbin, and every virtual file in /proc. Read the manpages, other documentation, and source as necessary.
If you aren't comfortable reading a man page or reading the source, that is your hidden feature.
From carlito -
I'm not sure if I am just displaying my ignorance but I just found out about the "last" command for check who has been logging in. It is very useful.
Another good program is "expect". I makes it extremely easy to automate any ssh/telnet logins or anything that forces you to provide input to a program.
raspi : Using SSH keys to do automated logins is much more preferred -
od
- dump files in octal and other formats. Useful to check for example if there's some BOM crap in the beginning of filefile
- what might be the filetype of given file?lshw
,lsusb
,lspci
- list hardwaretracepath
- are we dealing with MTU problem here?netwox
contains 200+ network related toolsip
- for network config/infosysctl
- filesystem/network/kernel infoebtables
- iptables for bridgevconfig
- VLAN configurationbrctl
- Bridge configurationsocat
-netcat
on steroidsipgrab
-tcpdump
-like utility that prints detailed header informationdig
- What's up with DNS server now?
From raspi -
tee
is awesome. Output to screen and logfile? Check.From nwahmaet -
I've always liked "man hier" for helping people that are new to the system.
From Matt Simmons -
I like the debian-goodies package:
Description: Small toolbox-style utilities for Debian systems These programs are designed to integrate with standard shell tools, extending them to operate on the Debian packaging system. . dgrep - Search all files in specified packages for a regex dglob - Generate a list of package names which match a pattern . These are also included, because they are useful and don't justify their own packages: . debget - Fetch a .deb for a package in APT's database dpigs - Show which installed packages occupy the most space debman - Easily view man pages from a binary .deb without extracting debmany - Select manpages of installed or uninstalled packages checkrestart - Help to find and restart processes which are using old versions of upgraded files (such as libraries) popbugs - Display a customized release-critical bug list based on packages you use (using popularity-contest data)
and also moreutils, which is basically awesome pipe tools on wheels:Description: additional unix utilities This is a growing collection of the unix tools that nobody thought to write thirty years ago. . So far, it includes the following utilities: - sponge: soak up standard input and write to a file - ifdata: get network interface info without parsing ifconfig output - ifne: run a program if the standard input is not empty - vidir: edit a directory in your text editor - vipe: insert a text editor into a pipe - ts: timestamp standard input - combine: combine the lines in two files using boolean operations - pee: tee standard input to pipes - zrun: automatically uncompress arguments to command - mispipe: pipe two commands, returning the exit status of the first - isutf8: check if a file or standard input is utf-8 - lckdo: execute a program with a lock held Homepage: http://kitenet.net/~joey/code/moreutils/
From jldugger -
ssh-copy-id for transferring ssh keys. The old way was to cat the key over ssh and even older was to scp the key and then cat the key. If you are using a non-standard ssh port then this will do the trick for you ..
ssh-copy-id -i /path/to/key '-p nonstandardport hostname'
Other wise..
ssh-copy-id -i /path/to/key hostname
From bobbyrcox -
find <blah> -exec <blah> {} \;
e.g.:
find . -iname *20080[123456].log -a ! -iname *.bz2 -exec bzip2 \{\} \;
Coops : -exec is useful, but "xargs" gives better performance. More details of the comparison here: http://dmiessler.com/blog/linux-xargs-vs-execniXar : Important: for find+xarg, always use find with -print0, and xargs with -0. Otherwise you can have extremely nasty surprises.From Jason Tan -
To change to the last directory you were in:
cd -
MathewC : I use this all the time.From dkaylor -
bash history - I normally have 10 or so xterms running. Here is the relavent part of my profile:
# Make history ignore dups, ls, and exit export HISTIGNORE="&:ls:[bf]g:exit" # Save 100000 history comamnds export HISTSIZE=10000 # Make each terminal use a separate history file HISTDIR=${HOME}/.history SHELLID=$(tty | sed 's!/!.!g') HISTFILE=${HISTDIR}/history${SHELLID} touch ${HISTFILE} # load last histfile as current history history -r $(/bin/ls ${HISTDIR}/history${SHELLID} | /usr/bin/tail -n 1)
If I need to search for a command, I can just grep through my history directory.
aespipe - reads from standard input and writes to standard output. It can be used to create and restore encrypted tar or cpio archives.
-
The power and flexibility of SSH never cease to amaze me. Also the ability to make a raid array out of anything (floppies, anyone?), all the filesystems available, cool stuff like LVM, the crypto tools, the possibility of crafting your own tools with the myriad of compilers, interpreters, languages... Oh, and not forgetting getting new stuff installed with apt-get or similar.
It is just so much brilliant than windows.
-
lsof -i - list all opened socket htop - like top, but with more eye candy rdiff-backup - for incremental backup mc - midnight commander ethstatus - ethernet statistics netstat, nmap iftop - display bandwidth usage on an interface by host
From miHost -
Not really a Linux thing, more of a Bash thing: process substitution,
diff some_local_file <(ssh somehost "cat some_remote_file")
This diffs a local and a remote file retrieved via SSH in one line.
From Rog -
NX combines some of the benefits of screen and ssh -X with clever compression. Now I can run gnumeric remotely over a modem connection and resume it when the connection dies.
niXar : Could you give an example on how to use NX as a straight replacement for ssh -X? I don't care for the remote desktop thing, just want to be able to run short-lived apps over a slow connection. The desktop thing is such a bear to config.From Nick Russo -
I see the
yes
command very nice:yes | do_you_agree
From man page:
NAME yes - output a string repeatedly until killed SYNOPSIS yes [STRING]... yes OPTION
From FerranB -
fgrep
e.g. fgrep -r * --include=*.rb
walks the dir structure from the current directory looking for in ruby source files
and is much easier to use than find
ptman : [ack](http://betterthangrep.com/) is another interesting tool for searching in files. It ignores version control files (like .svn -directories) by default and you can tell it to only search in ruby source etc.From Jeff Leonard -
I simply can't do without these commands
- ps -fA (list all running programs
- lsof (list of open files per process)
- pushd . (push current directory onto the directory stack)
- popd . (pop current directory from the directory stack)
- local account home directory ~/ you can also access a user directory like ~username.
- replace current bash profile after you've made changes without logging back in. ". ~/.bashrc
- grep -inR (recursive grep, i = case insensitive, n = show like number, R = recursive)
From Matt -
logger
lets you write messages to syslog from the shell prompt or a script.logger "See? There! It happened again!"
From Dennis Williamson -
dd
- convert and copy filesI use this very frequently to make copys of DVD's or HDD Patitions that I need to backup.
dd if=/dev/sda2 of=/tmp/copy_of_sda2
it's very handy and configurable, just have a look at it's man page
From cb0 -
POSIX Capabilities & File POSIX Capabilities
http://www.friedhoff.org/posixfilecaps.html
From Jeff Clark -
For "hidden" feature as in most often overlooked or missed by users new to Linux (/Unix).
man man
man -k <blah>
orapropos <blah>
"Hidden" features
From mctylr -
info, man, sed, awk, ngrep, nmap, netsed, tcpdump, strace, git/svn, ssh/scp, which, cut, dstat/htop/atop/mytop, dnstracer/host/dig
From Ruy Rocha -
<ESC> + .
'Pastes' the last parameter from the previous line into the current prompt
e.g.
ls -l /home/someuser/somedir/somefile
followed by
rm <ESC> + .
translates to
rm /home/someuser/somedir/somefile
From Bryan -
ctrl + r : searches history for the last command with the letters you specify
From Prozaker -
Let's see ...
I use bash color codes in my /root/.bash_profile to make my root bash prompt a different color than my non-root standard user prompt. Just one more reminder that I'm a single command away from destroying an entire system. ;)
I use nc (netcat) a lot to test things. Nice utility with a lot of versatility.
For the longest time I never knew that chown could accept both username and groupname together as an argument, i.e.:
chown user:group -R /some/directory
. That fact has saved me a lot of typing (no more chown/chgrp pairs).The df command's a quick shortcut to see all your mounted filesystems.
I use pgrep a lot in scripts to see if a process is running.
kexec is pretty neat, lets you reboot without going through BIOS, which shortens reboot times significantly. Don't really reboot a lot, though...
-
A post that I have made on Stack Overflow: How to list only subdirectories in the current one?
ls -d */
It's a simple trick, but you wouldn't know how much time I needed to find that one!
From edomaur -
find
is my greatest tool. I use it to locate things, and execute commands on files or directories found.No system is complete without
ssh
too.From Felipe Alvarez -
Never used script(1) to save a terminal session?
apg(1) to make random passwords
do you want to know how many processors/cores you have? nproc(1)
sfdisk(8) great partition tool
multitail(1) multiple tail at once...great toolFrom Pier -
Just found this one today:
dmidecode:
Reports information about your system's hardware as described in your system BIOS according to the SMBIOS/DMI standard. This information typically includes system manufacturer, model name, serial number, BIOS version, asset tag as well as a lot of other details of varying level of interest and reliability depending on the manufacturer. This will often include usage status for the CPU sockets, expansion slots (e.g. AGP, PCI, ISA) and memory module slots, and the list of I/O ports (e.g. serial, parallel, USB).It definitely makes answering questions about somerandomserver27 at the colo facility a lot easier!
wazoox : You'll like lspci, lsscsi, lsusb too :)gharper : Yup, I use the ls* commands on a regular basis - I just didn't realize there was anything that could pull BIOS level info :)From gharper -
For me, the greatest feature is that nothing is really "hidden" - it's all there right under your eyes if you bother to read the documentation or code.
From vwegert
0 comments:
Post a Comment