Front Page  |   Information  |   Computer  |   Computer setup tips  |   Router setup tips  |   Funnies & Everything Else  |   XP setup tips  |   Address munging for newsgroups  |   Junkware  |   Alternate Data Streams

Do you seem to be getting spam from my domain? Please see this note I'll put a after something when it's added to the page. After the item has been on the page for about a month, I'll take out the and just leave the "Added" date. At least I'll try to do it every now and then. If you find a dead link, a typo or have a suggestion, there's a link at the bottom of the page that you can use to send me an Email. Date format is MM/DD/YYYY

Do you use Computer Associates' ETrust Anti-virus? Even if you don't you should be concerned about Alternate Data Streams on Windows 2000 and XP. ETrust has created thousands of unnecessary files on my drives, and they haven't offered any solution yet. So I did some research. I like and use Firefox. Which ever browser you use, make sure you keep it updated. That goes for all critical software.

After several years of use I recently uninstalled Computer Associates ETrust Anti-virus when my license expired and (because I try to support companies that provide Free versions) I switched to AVG's paid version on that computer. They also have a free version, which I use on another computer. Just after that I heard about an Alternate Data Stream (ADS) scanner named LADS. When I ran it on one of my drives I was shocked to find that I had over 17,000 hidden objects that ETrust had apparently placed there as part of it's virus scanning procedure. They're "Alternate Data Streams", with one associated with every file on my NTFS partitions. I assume they contain some sort of hash of the file so that the AV doesn't need to rescan if the hash hasn't changed, but I don't know that for sure. The Streams all have CA_INOCULATEIT as part of the name, so it's obvious where they (almost certainly) came from. Because of their potential for abuse by malware, any decent malware scanner needs to scan each of them in addition to the normal files. All 17,000 plus in my case, doubling the time and CPU cycles necessary to do a complete scan.

That got me looking around for a solution. Naturally I contacted Computer Associates, thinking that surely they'd have a method of removing their [and only their] tags. Well, I tried to contact them. You can read the resulting rant at the bottom of this page. Update It's only fair to note that thanks to an outside party I was put in touch with a manager at Computer Associates who was very helpful, and tracked down a solution for me. Details are in the updates section below.

Since it looks like I'm not going to get any help from Computer Associates I started doing my own research into ADS and how I can fix my problem.

To be fair, I recently had a chance to run a test on someone else's computer who is still using Computer Associates Anti-Virus program and it has no ADS on it. At least none related to Computer Associates, just a few that Internet Explorer had used. They started using ETrust after I did (on my recommendation) and had updated to the latest versions more often than I did. So I don't know if they never had a version that used ADS or if one of their upgrades removed them. I'll be trying to find out.

After some poking around the internet and with the assistance of some of the participants of the GRC newsgroups (particularly Mark V. who knows far more than I do about NT file systems and related stuff] I've got some links to information on Alternate Data Streams and some programs that will find them, and in a few cases, remove them.

An Alternate Data Stream is sort of a file. All files and folders on NTFS are Streams of one type or another. The primary file is actually an UN-named Stream. Alternate Data Streams are named streams and have to be associated with conventional files. To access them the fully qualified name must be used. To open a text ADS in Notepad the command would be "notepad FileName:StreamName.txt" ADS can not exist on FAT formatted drives, only on NTFS drives. Alternate Data Streams can be any size and any type of file/data, and there can be multiple Streams associated with a single file. And any type of file/data can be in a stream and it can be associated with any type of file. So a program file could be hidden in a stream associated with (as an example) an image file. Microsoft created them to provide compatibility with files from the MAC OS. The problem is that they are hidden from normal methods of file inspection in Windows. They won't show up in file lists in Explorer or if you do a DIR from the command prompt, and Windows doesn't include their size in any typical calculations that it does. And of course any time that you have files that are hidden from Windows and it's users, you have the potential for malware to take advantage of it. There are reports of some BHO's using ADS to store their information and additional files. Viruslist.com briefly mentions one example of a virus using Alternate Data Streams to spread across multiple drives once an infection is in place, there are others out there too. There's even a remote exploit of some software recently reported by Secunia that takes advantage of a vulnerability in the way ADS is handled. I don't know if it's possible to use the same type of exploit in other circumstances, I imagine we'll find out eventually.

It's possible to have a zero byte file that has multiple streams of various types associated with it. So you can have a file that appears to take up no space other than it's file listing, when in fact it might have several large and hidden ADS files associated with it. If you find that you have unexplained zero byte files in unusual places it would certainly be a good idea to use one of the stream scanners listed further down this page to make sure that they aren't just 'place holders' for Alternate Data Streams.

While ADS can only exist on an NTFS drive, they can be accessed, read, and (apparently) run from a non-NTFS drive. So a program on a FAT32 drive can use an ADS as long as it is running in an OS that can access NTFS drives. ADS are apparently also backed up, at least when using MS's Backup programs. This means that while you can't copy a file with ADS to a non-NTFS drive and have the ADS survive, it appears that you could run a backup to a FAT drive, then restore it to NTFS and the ADS would survive. Probably. Some compression programs may preserve ADS, but I tested WinZip and IZarc and they don't, even if the zipped file remains on an NTFS drive. That doesn't mean that it's impossible for a virus/trojan writer to find a way to compress a file and it's associated ADS and send it by email. If your compression program isn't "aware" of ADS though, it probably wouldn't unzip them even if you were foolish enough to unzip the file itself, they'd simply disappear. I think, I'm certainly no expert. The good news is that it's apparently impossible for ADS to be included in a normal file attachment and be sent through email, it will be stripped off.

Keep in mind that those ADS won't show up in a file listing, aren't accounted for in file size calculations and can't be seen from any native Windows application. And while many operating system files are protected in Windows 2000 and XP by System File Protection, it's possible to add Alternate Data Streams to those protected files and not have them detected. So you can have ADS in critical Windows system folders and Windows won't make any noise about it at all. Fortunately programs that are hidden in ADS can be seen when they're running if you know where to look and what to look for. It's far from obvious though.

Not only can all that add up in terms of (hidden) used space on your drive, but Windows needs to keep track of each of these ADS, even when Windows won't show them to you. That can take up a lot of extra space in the Master File Table or MFT. And as they come and go it can cause fragmentation both on the drive and in the MFT, which can have a significant effect on the computer's operation. ADS can also be associated with folders, not just the files in them. The good news is that an ADS will not run just because the associated file is run, they have to be started specifically. But that's easy for malware to do once they've managed to get them on your drive.

There are a couple of issues regarding the ADS related programs I've found. Many of them won't list ADS associated with folders [or directories if you prefer], just those associated with files. Only a few of them will show ADS associated with some of the System files, "like System Volume Information\tracking.log". LADS will list them, and Streams will clean them though. As far as I know none of them will clean an ADS that has the Read Only attribute set. And the attribute is apparently inherited from the file that the ADS is associated with. Some of those files may need that attribute, so if you change it to allow the deletion of the ADS, then it needs to be changed back afterwards. On the files I tested changing the Read Only attribute on the parent file allowed the ADS to be deleted. But since ETrust has an ADS associated with every file on my computer, including all the system files (many of which need to be Read Only), that means a lot of extra work for manual deletion. I've also found one ADS scanner, SFind from Foundstone.com, that loops and starts over when it comes across a file that it can't access, which makes it fairly useless, at least on my computer. In my case it's a test file that my Anti-virus locks, and apparently that throws SFind for a loop. Literally.

For a very thorough explanation of Alternate Data Streams you can download the PDF file Alternate Data Streams: Out of the Shadows and into the Light from www.giac.org, which is associated with www.sans.org.

Frank Heyne has some good information on ADS in his FAQ.

Kaspersky used to use ADS to store hash data for files, but they stopped using them in early 2006. According to the press release it wasn't for security reasons, it was to speed up the removal of their software since it took a lot of time to remove all the ADS. However they also have an earlier article about the danger of virus writers using ADS for malicious purposes. And one problem with AV companies using them to store data is that the sheer number of ADS could hide malicious files using the same technique. I know that I would certainly have trouble picking anything unusual out of the mass of Computer Associates ADS on my computer.

According to Microsoft's KnowledgeBase Article Q101353 which was last reviewed in November of 2006 (as of this writing) Windows NT is inconsistent in it's support for Alternate Data Streams, so detectors, cleaners and other ADS related activities may not always do what's expected. I'm not sure how much of that may have carried over into NT's descendants, Windows 2000 and XP. KB105763 is more recent, although less recently reviewed, and it indicates that at least some aspects may not have changed.

Microsoft's KB Article 319300 mentions that The Windows 2000 Content Indexing Server adds an ADS that contains a thumbnail to image files.

And Microsoft's articles 883260 and 889815 have information about how IE stores origin data on files downloaded from the internet. This is what generates those warning when you try to execute a file that was downloaded using IE. There's only indirect references to it, but if you know what they're talking about you can see that the data is stored in an ADS. And other articles on the internet support that. So if you have a file that is bugging you with warnings every time you use it, you can use any of the ADS removal methods listed further down this page to get rid of the warning.

forensicfocus.com has an excellent article on Alternate Data Streams, with links to programs and more information.

infosecwriters.com has a good article titled The Dark Side of NTFS (Microsoft's Scarlet Letter) with links to programs and more information.

DiamondCS has an old but as far as I know still valid explanation of ADS and their risks.

A more recently updated page with lots of information is at bleepingcomputer.com. It even tells how to remove them. But none of the methods are practical for system files unfortunately, and all would be extremely tedious for the number of ADS that I'm dealing with.

LADS from Frank Heyne Software will list all your ADS files. LADS is a command line program, but LADS will not delete ADS files from the drive. LADS does have a filtering option that you can use to not show specific ADS. In fact he uses the Computer Associates ADS in an example. Using this command line for his program will scan the C: drive, including subfolders, and will not list any ADS with the string "CA_INOCULATEIT" in the name: "LADS C:\ /s /xCA_INOCULATEIT"

ADSSpy from Merijn is a GUI based program and it will delete Streams. You can use it to find all ADS, then select the ones you want to remove by selecting them from the list. There are several options like Select All and Invert Selection available on a right click menu.

Streams is from Mark Russinovich of Sysinternals.com, now part of Microsoft. It's a command line tool and can be used to find and delete streams.

HijackThis also from Merijn will find and clean ADS.

If you go about half way down Frank Heyne's FAQ you'll see a link to a Microsoft page that provides a download link near the top for NTFSext.exe. Download the file and follow the instructions on Frank's page [or other sites if you search] on how to add a tab to the file Properties dialog that will show you Stream information on files. It won't help you find files with streams, but it will help you get information about them once you know which files to look at. I've added some setup details at the bottom of this page.

Computer Associates does have a tool that will delete the Streams created by an earlier version of their program. And only their Streams, which helps considerably. However there are some hoops to jump through to download and extract it, and it has a couple of minor issues that may still be resolved. See my notes for some tips. Update: They have modified the tool to fix the problems I found, and put it on a page with a direct link. You should probably still look at my notes but the tool and some information can now be found at Etrust.com. I haven't had time to do thorough testing, but it appears to clean almost all the CA ADSs. In brief testing it only left one ADS on a file that most of the other tools couldn't even see.

Stream Viewer requires an installation to run and looks very similar to Microsoft's NTFSext.exe. According to the author's page it may even require lowering some of your security settings to run, which isn't something I normally recommend. I haven't tried it. But he has some very good information about ADS on his page. Stream Viewer will show Streams associated with folders, and it will remove streams.

crucialsecurity.com has an ADS tool, but I haven't given them the information they want to allow me to download it. In my opinion they want too much information for a simple download of a basic tool and a write-up. I'm told that it's a simple GUI program that lists ADS on all or specified drives, and has no option to delete them. At least that was true for the version available in early 2005. I'm not going to jump through the hoops to see if it's changed.

LNS is another command line tool for finding ADS. It does not have the capability to delete them.

Stream Explorer requires an installation and lets you browse files and their streams in an interface similar to Windows Explorer. It does not delete ADS. But you can actually view the content of the ADS. That can also be done with NTFSext (see above) with a little extra work.

DiamondCS has a tool that will tell you if your drive is capable of supporting ADS. You don't really need a tool for that, if the drive is NTFS it supports ADS, if it isn't it can't.

StrmExt.dll is the critical file for the basic functions. That file needs to be removed from the extracted list of files, placed in the System32 folder and registered. Two other files may be required for additional functions, but I haven't tested that. Mark V provided some nice instructions and comments which I've included in a separate text file.

It seems that ETrust AV only added ADS if someone enabled "Incremental Scanning". This was a feature in a rather old version, probably around the year 2000. I obviously thought that it seemed like a good idea at the time and turned it on, and I'm reasonably certain that ADS weren't mentioned anywhere at the time. The Stream contains a time stamp, a scan result and the engine/signature version that was used for the scan. On the next scan the ADS was checked first and matched to the file information. If there was no change, the file itself didn't need to be scanned again. That was eventually dropped when the driver's cache was improved.

So they created a utility to delete the Streams. All the references I can find to it on their site are dated 2002 and 2003. It looks like at some point the tool was included as part of a new version of their software and must have run automagically after an upgrade, resulting in removal of the ADS from most users' computers. But they apparently failed to include it on the CD for version 7.0, so they put it on their web site, which means it's available for download. It's in an [apparently] proprietary zip file with an extension of CAZ. Download the "QO40477.CAZ" file listed at the bottom of QO40477 and save it to a folder on your computer. Then download CAZIPXP which is the extractor. Instructions on how to use it are on that page. It must be used at a command prompt and there are no error messages if you make a mistake like mistyping the name of something. Once you have delstrm.exe extracted you can use the example at FAQ287184 to run the cleaner. The cleaner also needs to be used at a command prompt. The command "delstrm -V C:\ CA_INOCULATEIT" would remove most of the Computer Associates Streams on the C drive. I say "most" because it will not remove Streams from Read Only files. And it won't state that in the screen display that results, it will simply say that no Streams were found in those files. There's also no log output, so if data scrolls off the top of the command prompt window it's lost. FAQ282104 contains a bit more information about the CA cleaner and their use of Streams.

What I want is a tool that will allow someone to use a filter so that they only find specifically named Streams, then delete just those Streams, including any that are associated with Read Only files without changing the attribute on those files. I could do it myself with a combination of directing LADS' output to a text file, then using some sort of scripting to have that file run Streams to delete just those ADS. My problem is that I probably need to clean a bunch of other computers belonging to people that I recommended ETrust to. Some of those I'll need to do over the phone.

I've contacted Frank Heyne and he (understandably) has no plans to update LADS so that it can delete Streams using a filter. Someone else contacted Mark Russinovich on my behalf and he will keep my request in mind if and when he updates Streams but isn't making any promises. I plan to try and contact Merijn as well, but I imagine it would be a lot of work to add a 'delete filtered Streams only" option to his program. Update I never did contact Merijn because with the help of a manager at Computer Associates I located their cleaner (listed below) which can be used to remove just their Streams. See the notes at the bottom of the page for instructions and warnings.

I expect to do a little more experimenting just to confirm that my plan will work as intended. Then I'll use at least two programs to clean up my computer. I'll use LADS with the exclusion switch to identify all the ADS that aren't related to Computer Associates and save the list to a file. I'll also use LADS without the switch to get the total number of ADS on my computer. I'll also use ADSSpy to find all the ADS on the computer. I've seen a couple of discrepancies between the totals reported by different programs, so I'll use at least 2. Then I can use the tool from Computer Associates to remove most of their ADS. That will leave only Read Only files to deal with, which I can do as time permits. I'm more concerned with the number of the files and their effect on the computer than I am with the specific ADS. I'll use LADS and ADSSpy after Computer Associates' tool to get a comparison and see how many were removed.

Before doing any ADS cleaning be sure that you have a good back up available. A full image is advised. Remember that if you restore the backed up files, they may also restore the ADS if the backup program is streams aware. I use DriveSnapShot for my backups, and I've tested it, it does preserve ADS if the file is copied back to an NTFS drive, which is actually a Good Thing. Once you succeed in cleaning the ADS you'll want to defragment your drive. It would probably also be a good idea to do a chkdsk before and after cleaning the ADS.

Trying to contact Tech Support at Computer Associates turned out to be an exercise in futility. First I used their support page to open a ticket. After a few days I got an email response telling me to use one of their web pages to open up an email support request. Um, that's what I already did? Plus the link was misspelled and even after I fixed it one of the form fields on the page wouldn't work so I couldn't submit the form. 2 hours after the first email I got an automated follow-up email thanking me for contacting them and telling me that my support ticket had been resolved. Oh really? There was a link to a Satisfaction Survey which I gladly filled out. They must not be paying any attention to the results. I also sent an email to the address included in the body of their original response, and 12 days later got a reply which didn't quote my email so I had to look it up. That response told me to call a toll free number that got me to Enterprise Support. Except that they wouldn't talk to me, I needed to call Consumer Support. When I did there were only 2 options. If I'd paid for a support ticket press 1, otherwise go to the web page and open a support ticket or use their chat option. I already had a support ticket, so I tried their chat option. After filling in all the information on the form, when I pressed Submit I got a download prompt to install and run an exe file. There was absolutely no description or information of any kind about what it was, what it would do and what it might leave behind. For all I know it would have required me to use IE as well. Since I was already having trouble cleaning up after one of their programs, I declined. Other companies have managed to use chat type support using features built into the browser, why can't Computer Associates? Every time I use their pages I have problems because they require things that I normally block for security reasons. Some functions of some their pages don't seem to work at all using a browser other than Internet Explorer. That all seems a bit counter-intuitive for a security oriented software company.

A few days later I tried again, opening a new support ticket and explaining what had happened last time. Again the response said to call the same toll free number that wouldn't talk to me last time. But I tried again anyway, explaining my situation to the person that answered, but he said he couldn't help me. So I tried Consumer Support and pressed 1 this time. I explained the situation but he couldn't help me either since I hadn't paid any money to talk to him. He did finally give me another web page to go to that he said would open an email only support ticket. A few hours later tech support responded with an email apologizing because I had reached Consumer Support and telling me that I needed Enterprise Support. He referred me to the same phone number I'd gotten earlier. I sent back an email explaining again what I needed and what happened so far. I'm waiting with great anticipation for a response. I also filled out the new Satisfaction Survey linked in the email I got an hour later, thanking me for contacting them and telling me how glad they were that they were able to Resolve my issue!

I've exchanged several emails with the Product Manager. Computer Associates does have a tool (now listed in my ADS tools section) that will remove the Streams created by an earlier version of their ETrust Anti-virus program. I'm waiting to hear from someone involved with writing the software since I've found a couple of issues with it. See the notes below.

Where does the time go? Within a couple of days of the previous update Computer Associates had fixed their tool so that it will clean their Streams from Read Only files. It's now available on their support site without jumping through the hoops I described previously. I have a link to it in my ADS tools section. I did some quick testing that verified this, but haven't had time to do a thorough trial on my System drive because I want to do backups, scans, analysis, then clean it, then do more scans and analysis. Hopefully soon. But to be fair to CA I wanted to add this update. I have to say that the response in this case was outstanding, far better than my previous experiences.

I used the Computer Associates tool to remove essentially all their Streams from my computer. There appear to have been no problems at all.

The good news is that I've learned a lot from all of this, and hopefully this page will help out others who may have questions about ADS, especially the ones created by an older version of ETrust.