Sunday, June 13, 2010

Linux .. Infected

As I stated in my recent article about security, Linux is just as vulnerable if not more so than any other platform if not configured properly.  Yesterday, it was announced that a popular software package for Linux distributions contains a backdoor giving full access to execute commands as a user on the host where this software package is installed.

To quote the author of the forum post announcing the backdoor:

"It appears the replacement of the .tar.gz occurred in November 2009 (at least on some mirrors). It seems nobody noticed it until now."

The author also indicates that this issue doesn't exist in their code repository (CVS) indicating that it was most likely placed there maliciously by someone wanting to exploit users of the software.

This back door exploit has been out there for more than 7 months.   If this isn't an indicator of the need for better malware protection on the Linux platform, I'm not sure what is.  When are we going to admit that we aren't as well protected as we assume we are?  What other back doors are in open source that we just haven't found yet?  The number has to be greater than zero.

The most interesting part of this discovery?  Windows isn't affected.

"The Windows (SSL and non-ssl) binaries are NOT affected."

Signing source packages is a step forward, however unless we find a way to auto-magically verify the signature like some of our package managers do it won't help much.

The yet unanswered question though is what exploit was used to replace the original package last November?

NOTE:  Unrealircd packages containing malware were found in Gentoo and Arch, and were removed as soon as they were discovered.

31 comments:

Bobby said...

Don't get me wrong, but...so what?

A server was hacked, an archive was replaced with a modified version which contained a backdoor and this source package had to be compiled and installed by the admin. This doesn't tell anything about the security/vulnerability of Linux. And as far as I understand it, this backdoor wouldn't have been caught by virus/malware scanner either.

Instead the Linux permission management most likely prevented even worse on some servers, because the commands could only be executed with the rights of the user.

Also, Windows users *could* be affected if they used the compromised tar.gz to compile their versions (http://forums.unrealircd.com/viewtopic.php?f=1&t=6562&start=15#p32124).

FEWT said...

It had to be compiled by an admin, not the admin. Any distribution that released a package based upon that source is vulnerable.

Sure, it is restricted to execution under the context of the user executing the software but that user can still impact a lot of damage.

It can:

Create listeners above port 1024, it can make socket connections. It can access any local data that is not well protected. It can pick off keywords or inject a payload into connected IRC daemons. It can be used as a jump point to exploit other systems residing on or off the network that the infected system is connected to.

Windows users can be infected if they compile from the source, but that's really not the point. My point here was that only Linux was targeted when it would have been just as easy to target Windows users too.

P J said...

If a Windows user had installed the compromised tar.gz file it would do the same things to his personal files as it would to files of a linux user so no difference here.

nocturn said...

We need more mechanisms like SELinux and Apparmor that secure a system regardless of such a backdoor.

Malware scanning finds known theats only and enumerating badness is a futile exercise as the windows world demonstrates

Aaron said...

I agree that this isn't a big deal..and here is why. Is it in the Ubuntu repositories? Or any other distros package repos? If not..so what? Everyone should be careful with installing apps that aren't in the repos. Because someone is sharing a file that was modified/hacked and sharing it on their own website doesn't seem like a huge problem. Only install from safe places. On any OS if you install apps from all over you can't be assured of the security. If this was in the repos and/or has a large user base then I could see a concern...otherwise. /shrug

Saying that "Linux is just as vulnerable if not more so than any other platform." is VERY inaccurate. Please provide references to back up this claim. It's very well known that Linux is MORE secure than Windows. No one I've ever heard has said it's "completely" secure. But saying that is is "if not more so than any other platform" is propaganda and completely false. Period.

And anyway, windows users have a huge variety of virus/malware, etc. that can infect the OS. How many Linux virus's are there? Some, yes. But at least Linux is updated on a very frequent basis (for example Ubuntu). Security updates all the time, lots of people reviewing the code, etc. This method is far better than the windows 5-6 years with service packs every once in awhile that make the OS slower. Come on..

FEWT said...

Hi Aaron, thanks for your comment. Would you consider it to be a big deal if it was found in a distribution? Gentoo just released an update to remove the backdoor.

http://packages.gentoo.org/package/net-irc/unrealircd

I'm sure there will be others, I believe the package is also available in Arch. I haven't really looked to see if it was anywhere else.

The statement that Linux is just as vulnerable if not more so than any other platform isn't really inaccurate, as it challenges an assumption and not a fact.

I also have to challenge your statement that people review the code as this one existed for 7 months before being discovered.

I think this backdoor in addition to the screensaver trojan released a few months ago are evidence enough.

Sure there is lots of Malware for Windows I won't deny that. This didn't target Windows though, why?

Had the attacker released this for Windows in binary form would it have been found sooner? Definitely probable.

Jose_X said...

Let's compare.

Let's exercise our imagination:

Microsoft puts in 1000 backdoors into their software, either from install or after online updates.

Linux projects put in 1000 total backdoors into their software....

In which scenario do you feel safer?

Linux of course, because many many third party eyeballs go over the source, in particular, someone might have caught the violation within a small diff near the time it occurred (assuming the actual project people added the backdoor). Also, "I" can go and verify (eg, if I have doubts or want to be particularly careful in some use case). Believe it or not many people look over code all the time even if Smith Jones Jr does not.

Now, to greatly lower the chances anyone can modifies the code at some other point after a release, end users (administrators and people that compile source for self or for others) have learned to use signatures/hashes to verify. Obviously, not everyone heads this lesson, and in this case the project leads didn't either.

In real life, since Microsoft is the one that hides so much from everyone else, they have less pressure to close up their security code very tightly. In particular, it is helpful to their future sales if their software has holes that third parties learn to exploit as this forces upgrades once Microsoft stops supporting the old software. [Don't be like Maytag.] Of course, the problems with closed source like theirs goes further because their EULA and software allow for arbitrary information about your use of that computer and of files to be logged and shipped out onto the Internet. If their code was open, they would get away with much much less. When people can access the source, they take out anything they don't like.

gbin said...

Doh ! I was about to say that I am glad to be on gentoo with sources MD5'd automatically yada yada but the gentoo maintainer took the wrong tarball and this is really bad !

Anonymous said...

This is more of a reason to use GIT source code repositories rather than tar-balls or CVS. Then you have a signed history of all changes in source and no one can sneak in changes onto a hacked server. A server hosting the GIT repository could be completely controlled by a black-hat and it would be impossible (without the keys of the trusted coders) to introduce modifications to the code.

A trust chain relying only upon GIT for source and signed package binaries for distros nearly eliminates this back door injection. The problem it wouldn't eliminate would be a comprised trusted coder/distributor ( by them losing control of their private keys ). Or a trusted coder/distributor turning into a black-hat.

Anonymous said...

The simple fact that this is being pounced upon, as if this demonstrates ACTUAL SPREADING EXPLOITS, when compared to the Windows screen door, on their submarine, is expected and sad. Windows is that bad, with contrasting security.

FEWT said...

@Anonymous: I did no such thing I stated that it was a backdoor which allowed an attacker to springboard off of the host which is an absolute fact.

I did not in any way state or imply that this was a self propagating virus.

PatsComputerServices said...

IMHO, this doesn't reflect on whether one operating system is more secure than another. It does, however, emphasize that in the end it's YOUR responsibility to make your operating system as secure as possible.

In other words, I don't care what operating system you're running-- you need a good antivirus/antimalware system and a firewall. Because if you try to hide behind the operating system, eventually you will be burned.

The problem with Windows is about 100,000 viruses and worms were released for it BEFORE everyone woke up and realized they needed to do something to protect themselves (and before Microsoft woke up and realized that they need to do something to protect their customers).

Don't let that happen with Linux.

Anonymous said...

This does not show that Linux is less secure or even just as insecure as Windows.
And it proves to me that I'm safer running Linux than Windows. After all the backdoor has been discovered. How would I know what backdoors exist in Windows? How would I even begin to discover that?.
1. For all I know Microsoft servers were hacked long ago and backdoors inserted in Windows. Only Microsoft can alert me to this. And they have strong incentive to keep it quiet. So I have to take it on faith that there are no Microsoft or 3rd party inserted backdoors in Windows.
2.Linux is a diverse ecosystem, Windows is much more 'monoculture'. So cracking particular Linux servers may expose some Linux users, but probably not anywhere near most. Windows has a single point to crack/fail .. Microsoft.
3. If a backdoor does get into Linux as in the current case, I can fix it. With Windows I have to wait for Microsoft to fix it. And when they say they fixed it I have to assume that they did and that their fix did not introduce added vulnerabilities!

Linux gives you the tools to protect yourself. Windows says 'trust me, honest'. Which system is safer, more secure? The answer is quite clear to me. Linux

emk

Tim said...

This article is completely misleading. The "back door" was in an UnrealIrcd version that you neglected mention in your article and the fact that you "only" quoted someone else without giving the audience the entire picture... It's just bad blogging...

Before you go shaking your finger at the Linux community about security practices it might be a good idea to 1.) paint the entire picture and 2.) provide some basic guidance on how to avoid a compromised package.. Like running an MD5check sum against the package before you compile and install..

It almost looks like you intended it to be misleading..

FEWT said...

Hi Tim, there is a link in the article to the notice which includes all of the information you indicated that I neglected to post about.

It has been there since the first moment that I clicked "publish".

In-case you missed my other comment about the bad software being released in Gentoo (and possibly other distributions) you should look through all of the comments.

Concerning how to avoid bad software, I also provided a link to my original article that helps users be safer when using Linux.

Thanks for your comment, and for not really reading the article. :D

FEWT said...

Hi emk.

#1: Much of the Windows source code has been published to partners. These partners include security companies who's responsibility is to provide security protection to Windows.

http://www.microsoft.com/resources/sharedsource/windowslp.mspx

#2: Speculation

#3: If you are aware of it. Isn't the point here though that it was 7 months before it was discovered?

Both platforms give you the tools, but why wasn't this found for 7 months if Linux tools are so good?

Anonymous said...

..and while u are at it, someone owe to be taking a closer look at eggdrop deb builds, or cairodock. Seeing any unstoppable unknown processes lately??

Micheas said...

The reason it was not discovered is possibly because there are not that many people that run IRC servers.

I know several hosting companies that running an IRC server is a violation of TOS.

This is a little more news worthy than a Trojaned Gopher server, but not a huge amount.

The issue here is that Gentoo was hit by this, while FreeBSD was not, despite using the same mirrors.

Gentoo needs to look at their repository security.

A similar compromise happened to one of the FreeBSD mirrors years ago and it was discovered when people were complaining that the program would not compile, but if they deleted the download and tried again it normally would compile.

Nobody successfully installed the FreeBSD program that was compromised. The fact that there were compromised binaries from Gentoo means Gentoo needs to take another look at their security.

Patrick Aurience said...

I think that PatsComputerServices is the most on point comment so far. Regardless of your choice of OS, It is up to you to take your security seriously.

I don't think that the author at any time says that the Linux has the same number of vulnerabilities as Windows. Why does everyone have to take any news that their pet OS is vulnerable as a declaration of war? He noted that there is a vulnerability. It is a vulnerability that was distributed as part of a fairly common distribution and was not noticed for 7 months. With Linux you absolutely CAN look at the code. And apparently someone does, after about 7 months. Having the ability to do something and energy to do it are two different things.

The bottom line is, it happened. Learn from it. Windows didn't start out with worms that flew through the air landing on workstations from around the world. It started out with sneaker net viruses that didn't do anything other than make the PC speaker beep or flash a simple message on the screen. Instead of taking the Microsoft route and pretty much ignoring the phenomenon, (Or should I call that the Apple route) see that it is happening, acknowledge it and act to do something about it now instead of waiting until the horse is out of the barn with the barn burning to the ground like Microsoft did. This bury your head in the sand with your fingers in your ears shouting, "Linux is better than Microsoft," attitude that I keep seeing in these types of forums is sad. Show some maturity. It's as bad as watching a "liberal" and a "conservative" debate "health care". They both have pros and cons. I don't understand why both sides can't acknowledge that the other side has pros and their side has cons. Why does it have to be all or nothing?

I run Windows and Linux. I like both. I don't see a reason that I should have to choose one or the other. I like having options. But I also like to take care to make sure that my options don't get pwned.

Anonymous said...

Actually the comments here is quite telling. Instead of appreciating the information and learning from it - most see it as a personal attack on their beloved Linux! Come on Linux guys, grow up and take responsibility - learn from the weaknesses and who knows - the day may come that Windows will get some real competition :-)

Anonymous said...

One very dangerous assumption is that everyone developing for or hosting Linux distributions is trying to make it more secure, not less so.

If and when Linux becomes as mainstream as Windows there WILL be criminal activity creating exploits at source (pun intended).

If they're not doing so already ...

Anonymous said...

First off, thank you for the article fewt. Anytime someone rights about a Linux vulnerability you're bound to spend the next week or two fending off the wolves as you try to simply assert a point, well done.

I may have to agree with Tim about the overall context of the article being misleading. However this may be due more to laziness, rather the unwillingness by readers to read the links, than purposeful misconduct by the author. The author may have inferred, intentional or otherwise, a more damning picture of this particular case than truly existed in hopes of highlighting the true problem and motivating those that matter into action. This includes both system admins and repo owners.

Everyone that has posted thus far is probably more technically sound than I, and definitely has better grasp of english. That being said, I got into Linux because I was interested in computer security. I'm no expert, but I do know one thing when exploiting a system you exploit two things. Trusts and weaknesses. Even on a windows machine thats kept decently updated the best way is to exploit the trust of the weakest part of any technical system, the user.. ie find the biggest moron with the most control

I love Linux it lets me do what I want. As the admin I don't want a system that tells me, as the admin, what I can and cannot do as is the case with some windows systems. However this extra level of control comes with an extra level of responsibility. No useful system can protect itself from those that control it. As our user base grows it's important that those in charge of repositories take special consideration of the sources they use and trust. In addition admins home or corporate must be cautious when installing software outside of the repositories, assuming they do their job, buy finding trusted sites and checking the hash values.

Even then it's important to note that most windows vulnerabilities come from other software installed on the system. The guys from Metasploit mentioned in one of their speeches at the BlackHat conference that many times it's not an inherent security problem with the system but in-house software that provides a rout in.

Linux may provide another/harder barrier between the software running and root access; however it's not immune to exploits that allow others user access. Root access or remote shells are not the pay day for hackers today, its the information contained in that system that is valuable. Linux is no more immune to these things than any other system out there.

We are given the tools as an open community to freely exchange the ideas, the tools, and the know how to those less fortunate. If we are to succeed this is how it shall be done: learn, teach others, write better code, improve better code, and be aware of who/what you are trusting.

End rant.....

adamwill said...

I'm surprised no other commenter picked up your description of unrealircd as "a popular software package for Linux distributions", which is *really* stretching a point. One, it's not in many distributions, as we've already established. Two, it's a frickin' IRC server. 'Popular'? Really? How many systems, in the entire world, are running as IRC servers? I'd guess it's in the hundreds. Possibly - possibly - thousands. Three, it's not even *the most popular* IRC server. You're on pretty thin ice, with that line.

As others have pointed out, all you've heroically exposed here is a two-bit software project with really bad release procedures. Any half-serious project - especially one producing a server - provides signed / checksummed tarballs. Any half-serious distribution thinks very hard before including code which *isn't* signed by upstream, for exactly this reason. Especially server code. This is likely one reason unrealircd isn't in many distros.

This really doesn't engage with any of the safeguards which make Linux overall a more secure environment when used correctly, because it involved bypassing almost all of them. All you've really demonstrated is that any platform which allows the execution of arbitrary code is potentially vulnerable to compromise. Well, duh. Anyone with a vague interest in computers should know that already.

"The most interesting part of this discovery? Windows isn't affected."

As someone else has pointed out, that's not particularly interesting. All it means is that the attacker just modified the source and sent a compromised source tarball, they didn't take the trouble to provide compromised pre-built binaries too. I don't blame 'em, building from source on Windows is such a giant pain in the ass.

"Signing source packages is a step forward, however unless we find a way to auto-magically verify the signature like some of our package managers do it won't help much."

You managed to answer yourself within the sentence. We already have a perfectly good mechanism for doing this. It works very well. You're only in trouble when it gets sidestepped.

Also, please don't say 'auto-magically'. It's a horrible word that needs to die. Just say 'automatically', because that's what you mean.

"The yet unanswered question though is what exploit was used to replace the original package last November? So how can we fix this problem? "

Could be anything. How do you know it's even an exploit in Linux? They exploited the hosting provider, not the app in question. As far as I've seen no-one's even bothered to verify who the hosting provider is.

FEWT said...

Hi Adam, thanks for your comment. I found the package in Gentoo and Arch two reasonably popular distributions wouldn't you agree? I'm glad that the exploited software wasn't in Fedora since I am a Fedora user.

Concerning popularity, it depends. In the world of hosting IRC it is popular, certainly. Is it popular for desktops? No. Is it popular for database servers? Certainly not.

Honestly I haven't "exposed" anything really, it was already exposed when I found it. I simply provided an analysis, and opinion then asked a few questions. I completely agree that signing and checksumming tarballs is important as long as the signature or checksum is stored somewhere else, where it isn't vulnerable, and that they are checked.

I completely agree that anyone with a vague interest in computer should know that execution of arbitrary code is potentially harmful. This wasn't arbitrary code though posted in a forum.

I found it very interesting that the Windows binary had not been swapped because that implies that Linux was the target. Anyone that can exploit or find their way into a server, patch source code, and build a package that looks like the original source can build software on Windows. It isn't that complicated.

It is absolutely interesting that no Windows package was released because typically Windows is the target, only not this time.

Signing packages works well, sure so why shouldn't we discuss how to make it work better, or make it more foolproof or seamless?

Did they exploit the hosting provider or the host itself? It is a Linux host according to netcraft, I know this because I checked.

Anonymous said...

Gentoo and Arch aren't that popular. Taking distrowatch as a (admittedly poor) guide to popularity, you get Sabayon (Gentoo-based) at 8 and Arch at 9, so they don't rank too high. Why target Linux instead of Windows? That might just be because Linux is typically preferred as a server OS and it was a server package that was compromised.

Your post does raise a valid concern, that the source packages need to be signed, but it doesn't need to be automated because most people use distro-provided binary packages, and the package maintainers should be capable of checking the signature and even including automatic verification of the downloaded package as part of their build scripts for the package.

IMO, this isn't a big deal, but highlighting/reminding people of this will hopefully prevent this in the future.

Anonymous said...

"The most interesting part of this discovery? Windows isn't affected.

"The Windows (SSL and non-ssl) binaries are NOT affected.""
Then neither are Linux. The Linux binaries was also NOT affected.

So far the only reported affected binaries is Gentoo. However many users build their own binaries so there may be many Windows users out there that has the backdoor.

It is dangerous to write that Windows is not affected when that isn't true. Because of this it is possible that affected users stopped reading and think that they are not affected.

Anonymous said...

"Both platforms give you the tools, but why wasn't this found for 7 months if Linux tools are so good?"
Because the gentoo packager that distributed the infected version did not run the tools, he or she just shipped it without checking it. The site clearly states that a simple checksum validation would have shown that it was not the official source archive.

FEWT said...

Arch also released an update to remove the backdoor.

http://bugs.archlinux.org/task/19780?project=5

The only binary executable released was for Windows.

"For reference, here are the md5sums for ALL proper files:
7b741e94e867c0a7370553fd01506c66 Unreal3.2.8.1.tar.gz
5a6941385cd04f19d9f4241e5c912d18 Unreal3.2.8.1.exe
a54eafa6861b6219f4f28451450cdbd3 Unreal3.2.8.1-SSL.exe"

Sledge said...

I think the title for this article should have been "Linux Server Compromised" and not "Linux..Infected" since it was a bit misleading. The source code on the server was replaced but there were no sufficient information on how it was done, it could have been a through another vulnerability or the attacker could have simply gained access to the physical server and just copy and paste the file. Either case the real issue here was the server and not the package. I do agree that with this kind of attack, without a proper signature verification, any operating system is vulnerable. Another way of protection from this attack is by using a mandatory access control system like Apparmor and SELinux. I don't know if there is any equivalent software in Windows but Anti-virus and Anti-Malware are more like a reactive and not a pro active solution.

FEWT said...

I don't find it misleading at all. The source code that was replaced exposed a backdoor. It may not have involved every single distribution, however it highlights the absolute need for good security practices.

Neither SELinux nor Apparmour would have mitigated this issue.

Anonymous said...

Reading the original post, we see that the backdoor was placed in source code, not in binaries.
So, it is not targetting Linux, but every system that compiles the software.
This way, this is a real risk for anyone.