Wednesday, September 30, 2015

New Community TDS (NCETDS... just kidding we have enough acronyms!)

Video Tutorial - [ Here ]
Developer Documentation - [ Here ]
Community TDS inferface - [ Here ]

This blog post (one of the few by Andrew) is here to tell you about the new public TDS (technically an update for the community TDS so that it is inline with the private TDS source base). For those who aren't interested in reading all the words we have a great video to talk about this below:

Let's start off with an introduction to the TDS. It provides an easy to use, distributable means of writing and sharing transforms (and essentially the data so that users can turn that into intelligence) . All the transforms in the transform hub are built on either the the free public TDS or a private one.

When a "normal" transform (one on the public/private CTAS) runs what happens in the back is that a message is sent to the server containing the entity details (like its value and other properties) as well as the transform that needs to run. For example it could be the domain and the transform "to MX record". This would then be run on the server (the code would execute - performing an MX lookup on and the result would be returned to the client.

Previously people used local transforms which had a number of painful setup and distribution points:
  1. Local transforms require people to setup code and environments on the end user system.
  2. Code updating was painful as you would need to send all your users new code to run.
  3. Code containing all the API calls, passwords and other sensitive information needed to be obfuscated.
Our solution to this was the original TDS. Essentially what it does is provide a way that users can write and create transforms that they host on a web server. This is all done through a simple and intuitive web interface (the TDS).

What happens with the TDS is that when it recieves the call that includes the entity and transform to run, (as described previously) instead of executing code on the machine it will simply make a call over HTTP or HTTPs to a web server. This web server then receives the call and can then do whatever it needs to - be that talking to a database or API or something else... literally anything that you can write a program for.

You can read more about this over at our developer portal. It's got explanations, code samples and more. It will quickly get you up to date with the aspects of coding transforms.

This update of the community TDS keeps it in line with the private version with a range of new features include the following:

OAuth Integration

OAuth Integration allows transform writers to utilise open OAuth integration connectors (such as Twitter) or write their own to control who uses their transforms or just to do statistics.

Paired Configuration

Developers can now pair  exported Maltego configurations with their transforms which means they no longer need to ship MTZ files containing entities, machines, sets and seeds. These configuration files can be simply uploaded to the web interface and when the end user discovers the transforms they will automatically get these items added to their client!

Bug fixes, interface tweaks

A number of bug fixes and interface updates have been done to the interface and the whole experience should hopefully be more usable and intuitive for everyone :)

What are you waiting for? Head on over to the new community TDS  now!

Pink fluffy unicorns, dancing on rainbows

Thursday, September 3, 2015

Jumping on the Website Tracking Code bandwagon

Services like Google Analytics allow you to easily add functionality to your website simply by pasting a bit of JavaScript into your page's html. Often this JavaScript includes a tracking code that uniquely identifies the site owner's account with that service. Searching this tracking code with a search engine that indexes JavaScript allows you to find other sites that belong to the same user. There are quite a few web services that require you to add a tracking code to your webpage in order to use it. For analysts this provides a great way for making connections between websites that may seem unrelated using other OSINT techniques.

Recently there was an interesting project write-up called Automatically Discover Website Connections Through Tracking Codes by @jms_dot_py and @LawrenceA_UK. They used the source code search engine Meanpath to search for websites with a specific tracking code and Gephi to visualize the relationships from their results. We've been having the same idea for while now and decided to release two new transforms today. This means you can use this technique from within Maltego.

The first transform is called To Tracking Codes and runs on a website entity in Maltego. The transform will parse the home page of the specified site for tracking codes from services including Google Analytics, PayPal donate buttons, the Amazon Affiliate program, Google Adsense and AddThis. The image below shows the different tracking codes that can be found with this transform as well as the Detail View that is returned with each entity that includes a source code snippet of where the tracking code was found. The second transform is called To Other Sites With Same Code and is used to find other website that have the same tracking code.

Let's see what can be done with these transforms with a quick example using the Google Analytics code found on Ashley Madison's home page from the graph above. Running the transform To Other Sites With Same Code returns 100* different sites that all use a tracking code from the same Google account as the one from Ashley Madison. The resultant graph is shown below. (*Currently this transform is limited to returning a maximum of 100 results so there could actually be far more sites).

Most of these sites are just variations of the name and all redirect to Ashley Madison's home page. There are also a few other online dating sites here too as well as a couple of completely unexpected results of pages that you would not see being related to Ashley Madison in any way. These sites have piqued our interest so let's look a little deeper.

Taking all the websites from the previous step and running the transform To Tracking Codes again only finds one new code on the sites and Running To Other Sites With Same Code on this new code does not result in any new sites being found. This looks like it could be a dead-end so let's use another tool we have in the Maltego workbench. Resolving all the websites in the graph above to IP addresses shows that most of these sites sit on the same IP address except for a couple of outliers as shown below:

(only a portion of full graph)
We are looking for something out of the ordinary that is seemingly unrelated to Ashley Madison. We next remove all the sites with titles that are obviously related to Ashley Madison. This results in the graph below with just a couple of IP addresses that are scattered across the globe.

Finally let's see what else resolved to these IP addresses by running the transform To DNS Name [Other DNS names]. This transform will return historical DNS records for these IP addresses. Doing this results in some really interesting NSFW sites specifically found on the IP address that also host and

The image below summarizes the connection found between Ashley Madison and our somewhat unsurprisingly very much not safe for work (VMNSFW) websites that won't be listed here.

These two new transforms for working with website tracking codes are now available in the PATERVA CTAS seed on both commercial and CE. Simply hit the Update Transforms button in the transforms hub and they will be added to your Maltego client.

As always, enjoy responsibly,