Ads could either be inserted into a <div> (which used a blocking document.write call behind the scenes), or they could be inserted into an <iframe>. The majority of publishers only ever used the iframe method, but a few were provided with div insertion codes.

As our ad serving capabilities have advanced, we’ve created more and more formats which only work when inserted as an iframe (e.g. What we now call “Fun Units”).

At the same time, we have advanced our iframe insertion significantly:

• iframes are now loaded asynchronously on most web browsers – meaning your website loads much faster while our ads load in parallel.
• We’ve added lightbox ad formats, and extra library methods for ads which make our iframes super-powerful.

iframes brought many other improvements, such as fraud prevention and extra security for viewers from potential “malvertising” and “clickjacking” attacks.

We have now switched all ad unit embeds over to the iframe method.

You do not need to make any changes, and this should affect very few publishers.

However, if your current ad unit embed code contains a <script> tag pointing to this url:

http://syndication1.viraladnetwork.net/getad/?style=frame

Then you can now make your site a little bit faster for some users by changing that url to:

http://syndication1.viraladnetwork.net/getad/

All new embed codes will reference this new url, and the probability that it will already be cached in your user’s browser (or by a proxy) is higher.

According to The Guardian, the UK based Daily Mail has overtaken the New York Times to the most viewed online newspaper position.

The Media Blog explains that this has largely been achieved by the Mail Online focusing on scantily-clad celebrities, female figures in Bikinis, and other titillating content. This makes it’s readership quite different from the “white, middle-aged, middle class of middle England who worry about a news agenda of wheelie bins, cancer, hoodies, liberals and multiculturalism” that the Daily Mail’s print version caters to (as the Media Blog describes the audience).

But how much of a good example does this set for publishers?

Yes, the Mail has increased it’s readership very significantly – but the question is what has that done to their bottom line?

Very few brands would choose to advertise next to this kind of risqué content – meaning less competition, and lower prices. For those who are advertising, user engagement rate is likely to be lower – leading to lower click through rates.

A quick search for “Mail Online Revenue” brought the results I was expecting. Paid Content reported in July that the revenue per user had almost halved over two years – despite readership having increased by 82% in a single year.

After the overheads for serving content to this many viewers is included, it would be interesting to see how much of this reduced revenue they managed to turn to profit. Finding that out by itself would be difficult.

The Mail Online’s parent group (Associated Newspaper’s) reported a loss of $1,407,127.19 on digital operations in the year ending October 2011, which their CEO put down to struggling to sell ad space in the US, as well as capital spent on growth of operations.

The Mail’s $1.4M loss is compared to the NYTimes’ performance in Q3 alone, who posted digital revenue of $50.3M, and total net income of $15.7M across the company ($20M up from the year before) – despite declining revenue from print.

Perhaps the Mail Online isn’t setting such a great example to other publishers after all…

This is part of internet-wide action against the bills coordinated for Jan 18th, and we are not being paid to run these ads.

For more information, visit americancensorship.org

• Serving of ads underwent 10 minutes of downtime for approximately half of all users.
• Tracking of actions underwent approximately 35 minutes of downtime.
• We will be bringing non-core functionality (such as dashboards) back online slowly during the next hour or so.

Tim Wintle

Perhaps the most confusing comment was from Twitter. Here’s the email their PR teams have apparently been sending out to news outlets:

For years, people have relied on Google to deliver the most relevant results anytime they wanted to find something on the Internet.

Often, they want to know more about world events and breaking news. Twitter has emerged as a vital source of this real-time information, with more than 100 million users sending 250 million Tweets every day on virtually every topic. As we’ve seen time and time again, news breaks first on Twitter; as a result, Twitter accounts and Tweets are often the most relevant results.

We’re concerned that as a result of Google’s changes, finding this information will be much harder for everyone. We think that’s bad for people, publishers, news organizations and Twitter users.

Hang on a minute, are twitter implying that this change is going to stop twitter showing up in results?

As Google’s Matt Cutts blogs, the new change pulls in results from lots of sharing sites – including livejournal, friendfeed, and Google+.

You may ask why links from twitter aren’t on that list – is it some secret manipulation by Google? No. Twitter have told Google not to include results from Twitter.

How did they tell Google not to include their results? Well if you look at the HTML source for twitter’s website, you’ll see that all links from twitter have an attribute rel=”nofollow” on them. This isn’t an accident – it’s a special feature which Google introduced in 2005, designed to let websites tell Google not to show those links. There is no other reason for it – if you add that to your links (like twitter), you don’t want Google to count them.

There’s more though – you may remember that Google used to show more tweets in their search results, and why why that changed. The answer is that Twitter used to have a deal with Google to give them direct access to their data – which was not renewed when it expired in July 2011.

So in summary, six months ago Twitter stopped Google’s access to most of their data, and they tell Google not to include anything linked to from tweets. With that in mind, does Twitter’s statement about Search Plus Your World:

We’re concerned that as a result of Google’s changes, finding this information will be much harder for everyone. We think that’s bad for people, publishers, news organizations and Twitter users

still make sense? Or does it suggest that Twitter may have an different motive for their emails?

organise the world’s information

Google’s Mission Statement is to “organise the world‘s information and make it universally accessible and useful”. And I personally believe that this latest change really is a tactical move to this end, rather than a pure land-grab for Google+ in the social space.

While Twitter do provide APIs to access their data, they charge significant sums to get real access at the volume that Google needs it.

By opening up large screen volume to social sites who provide free access to their data (such as livejournal and friendfeed, and their own Google+), they’re providing a huge incentive for sites to make their data accessible to be indexed and searched by search engines – which is Google’s core business after all.

Here’s a sample of a chart I’ve been looking at, showing how many ads we were displaying per second over the festive period – you can ignore the different colours, they represent which datacentre ads were being displayed from.

It was the night before Christmas…

And all through the house not a key was pressed, nor even a mouse [button].

Yup – Christmas Eve saw the majority of web traffic drop off far earlier than the rest of the year, as people got a good night’s sleep ready for the big day. (And countries who have their main celebration on Christmas Eve settled down after a long day). Traffic during the night (at local times) was approximately 30% lower than normal.

Christmas Day

I for one was too busy putting the turkey in the oven and peeling veg to be surfing the net at the start of Christmas day – and with kids distracted by stockings and new toys, internet usage seems to have been focused in a much narrower period in the middle of the day.

This is different from the behaviour we used to see a few years ago, when there would be a large surge of traffic in the afternoon as people brought their new internet devices online. Perhaps the market in the countries we are focusing on is saturated enough that Christmas no longer brings so many new internet users.

Boxing day relaxation

Boxing day was the first day I had time to go online to catch up on emails etc – and it seems I was not alone. From early afternoon people came online in their masses – and many stayed there late into the night – making this boxing day come very close to being a record-breaking day for the Viral Ad Network – we served only 350K less ads on this Boxing day than on our highest day ever.

What did you do?

What was your internet usage like over Christmas? Did you spend Boxing day shopping online, talking to friends, or reading news sites? I’m interested in hearing – leave a comment below.

At first read my eyes read the CAPITALS, and then skipped to “ends Sunday” – leaving the message that Monday deals end Sunday…

It was only when I wrote this post that I spotted it said “Week” – and by that time I wouldn’t have been looking if I weren’t writing this.

One of the most common arguments justifying using tags carrying semantic value (such as <header>, <article>, etc) precisely is that it makes it easier for search engines to index your site…

But i’ve never heard someone who has written an indexer focus on that argument.

(note I’m not saying there isn’t a very minor effect… but certainly not a massive change)

When an indexer is trying to pick out the important text on a page, it has two main problems:

1. Removing all the unimportant text (headers, navigation, footers etc)
2. Dealing with spammers

The problem is that any technique websites can use to give hints to indexers will immediately be abused by spammers.

Take the keywords meta tag. Back in 1995, search engines thought it would be great if websites could add some semantic data to their sites, and they invented the “keywords” meta tag – which let websites specify keywords describing their site.

That failed badly – for every site which used the tag to add useful keywords, several spam sites used the keywords to add irrelevant keywords to their site. At the time it wasn’t unusual to skip through several pages of search results before finding a site which wasn’t spam. By 1998, search engines had started ignoring the tag.

As a result of spammers, it’s fairly much a given that all indexed data about the page has to come from what the user can see on the page – not from hidden markup features.

CSS lets us style elements in any way we like – we can effectively change the visual meaning of any two tags however we see fit.

It would be trivial for example to switch <h1> and <small> tags by modifying the styles for the elements. On a normal website you may never want to do that, but now consider you’re a spammer – you can put two different sets of copy on the page, and the raw markup can focus on one set of copy while your CSS focuses the user on completely different copy.

If an indexer was to base their importance largely on which tags were being used (e.g. rank <h1> tags highly) then they would be wide open to spam from that page.

It is possible for indexers to take hints from these tags, and use them if they match with their own results for detecting sections of a page, but they certainly can’t take them at face value.

We’ve now set our crawler live across all publisher sites, so it’s time for a formal introduction.

What is VANCrawler?

VANCrawler is a web crawler which downloads a copy of all pages showing our ad units onto our own servers, where we apply our own natural language processing for advanced contextual targeting, scan pages for potential issues, and run countless other processing to help us target ads better.

What Do I need to do?

We have spotted several sites which have issues with the text encoding used on their pages. We suggest that all publishers double-check that they are declaring the correct encoding (“charset”) for their site.

Without this information, crawlers don’t know how to treat words containing accented letters such as “publicité”, or non-latin letters such as “Διαδίκτυο” or “網站”. If you’re not using these letters you may still be affected, as without the correct character set we may believe you are.

This is a problem for crawlers which index pages for search engines, and for users around the world as well as our own crawler, so fixing this issue on sites where it occurs may increase your search engine traffic as well.

You can specify the character encoding for your website in the Content-Type HTTP header, in the XML declaration for XHTML, or using a special tag for HTML. Please ensure that this matches the character encoding which your documents use when they are served.

If you are uncertain which encoding you are using, you can generally find the “charset” specified somewhere in your web browser (e.g. under “View>Character Encoding” in Firefox. For European users using windows, this will probably be “Windows-1252″. For Users on any other operating system (or for advanced websites on Windows), it will probably be “UTF-8″.

We have also noticed several sites serving pages which are not encoded using a valid encoding at all.

This may occur from using a server-side language such as PHP to concatenate strings which use different encodings. For example a string stored in a database may be encoded using UCS-4 (where each letter is represented as 4-bytes), and may then be concatenated with an ascii string (where each letter is one byte). This situation is even more difficult for crawlers to deal with, and so we suggest that publishers who program their own server-side pages check that they are not mixing different character sets when generating their pages.

Can I prevent VANCrawler from accessing my site?

VANCrawler follows the robots.txt standard, and will follow rules for User-Agent: VANCrawler..

If you want to limit how often we crawl pages on your site, you can use the crawl-delay robots.txt option to specify the minimum number of seconds we should wait between crawls.

As VANCrawler is being used to target ads to increase your revenue, we recommend you do not disallow access to pages on your site, however you may enter urls to exclude in the robots.txt format.

You can find more information on our ad crawler here: http://www.viraladnetwork.net/crawler-info

We will update this post with more information as we have it.

Update: 16:40 BST

All Ad serving systems are currently back to normal.

Dashboards are currently offline while we allow ad servers to catch up.

Final Update

As of 18:15 BST all dashboards and analytic tools have been brought back online