It’s been a while since our last announcement, but we haven’t been sitting idle all this time. Here’s what happened on PagePeeker lately.
Moved everything to SSD storage
We have been using SSD storage for our front end servers for a while and it’s great for storing the database, website scripts and the operating system. But the thumbnails were stored mostly on magnetic disks, with only a small cache on SSD. This made the storage cost effective, while still allowing some speedup on the frequently used thumbnails and screenshots.
With the latest drops in SSD pricing, as well as the increase in business we are seeing, it started to make sense to use SSD storage for storing everything, including the thumbnails that are cached for longer periods of time. So we switched our servers to fully use SSDs and they have been working very well since then.
This had been implemented for both the free and paid service.
Also, if you noticed same short service disruption intervals, they have been caused by the server upgrades. Which were supposed to go flawlessly according to our plan, but, as sometimes happens, the plans didn’t work as expected..
Enabled image compression for thumbnails
We’ve been busy lately implementing a service for image optimization, which you can check out at http://shortpixel.com/,Â and we learned a lot about how to optimize images, both lossy and lossless. So we applied the lessons learned there to PagePeeker as well and reduced the thumbnail sizes by an average of 20%, while still keeping the exact same image quality.
This is currently being rolled out on the free service and will be gradually be rolled out to paid customers as well in the following weeks.
An overall faster service
Combining the SSD storage, that allows for faster thumbnail retrieval, with the smaller size of the thumbnails and with other networking optimization we have done on the servers gives us a better speed in delivering the thumbnails. We have reduced the average thumbnail serving speed from 250 milliseconds to just under 100 milliseconds.
How is the service working for you? Let me know in the comments.
Since last writing on the blog, there have been a number of important updates I would like to share with you.
Fully redundant infrastructure
We have changed out infrastructure to be 100% redundant. From the redundant load blanacers in front of the servers, to redundant frontend servers, to redundant screenshot capturing servers, everything is now failure safe. This way, we made sure our infrastructure has no single point of failure.
The only non redundant thing remaining is the hosting provider. We have all our servers hosted with the same provider, though in different datacenters. In the unlikely event the provider goes down, so will we.
More screenshot capturing servers
We have more than doubled our screenshot generating capacity, to keep us with the increased demand. This also allows for a lot more room to grow, and includes enough capacity to handle spikes in traffic. Even if we get a spike 10 times the normal size, we will be able to render all screenshots in a timely manner.
More features for premium customers
We have added a number of features available to premium customers:
- Callbacks when the screenshot has been generated. This allows you to pass in a URL that will be called when the screenshot has been generated
- Custom headers. This lets you configure what headers to send along with the requests. Handy for custom authentication schemes as well.
If you are interested in using any of the above features, do get in touch and we’ll have your accounts configured to use them.
Just a quick heads up: we are having some troubles with one of our servers. This causes a service disruption for some of our customers.
Our engineers are working to resolve the issue. The estimated downtime is two hours (until 12:00 EDT)
Will update this post as things are progressing.
Update 13:26 EDT:
- we experienced a raid failure on our primary storage server
- this dragged down our entire network so all our customers are influenced now
- our engineers are still working on restoring the service
- new estimate for the restoring of the service is 17:00 EDT
Update 19:00 EDT:
- services has been restored for most of our customers
- we are still working on bringing online the rest of our servers and restoring services to all our customers
Update 23:00 EDT:
- services has been fully restored for all our customers
I apologize again for all the inconvenience caused by this extended downtime. We have learned our lesson and will speed up the roll out of a fully redundant infrastructure. This has been in planning for a while and this downtime motivates all of us to roll it put sooner than planned.
We just released a new feature that has been on development for some time: PDF screenshots.
This is a transparent process for everybody using PagePeeker, both free and paid accounts. Our robot will understand that the URL contains a PDF document and will do a screenshot of the first page. Depending on the page aspect ratio it might look a bit distorted, but it will help understand what the PDF document is about.
Please test it out and let me know in the comments how you like it.
Starting today, we are making our fastest rendering speed available for our demo page.
Why did we decide to do that? So it’s easier for you to see how fast we can generate thumbnails from web pages. This has been available for a long time for our premium customers, and we have decided to make it available on our demo page for everybody to see how fast we are.
Here are a few notes about the faster version:
- Should work well for most of the sites
- Flash sites will almost never render properly. That’s because we don’t allow enough time for the flash movie to load and initialize. Our normal rendering method is slower on purpose so it allows time for it
- This is available as a demo only and can be made available for paying customers
We have not yet decided whether to make this faster version available for the free accounts too. We’re thinking it would be a good addition for our “free unbranded” package, bout would like to get your opinion as well.
Please write a comment if you want this for the free version.
We have implemented two new services and are currently looking for early adopters to help us beta test these. Later, we are going to launch them in open beta so they will be available to all customers.
Both services are available for premium accounts only, and will be made available free of charge for the beta testing period.
The first service offers the ability to screenshot PDF documents. This works both for above the fold screenshots and full length screenshots. A word of caution for the full length screenshots. We are going to limit this to the first 5000 pixels of the document, as larger documents can result in undesirably large images.
The second service is allowing the saving of the web pages as PDF documents. This should preserve the aspect of the site without resorting to capturing an image from it. This way, if copy / paste is required later, it will be available from the PDF document.
If you are interested in testing any of the above services, please get in touch with me via the contact page. If you have any questions, please ask in the comments.
We are proud to announce the launch of the TwitSplash.com site. This is a spinoff from our company and it focuses on adding a visual aspect to the twitter reading.
The site provides 3 kinds of enhancements to the twitter twits:
- Inline images. It shows the images embedded in the twits, with large previews
- Inline videos. The videos (from YouTube for now) are embedded inline, with a an easy one click viewing.
- Webpage previews. Shows thumbnails of the links embedded in the tweets, if they are not images or movies.
Obviously, the PagePeeker technology is used for rendering (very fast) the webpage previews. Without our ability to render website thumbnails so fast, it would have been impossible for twitsplash to even exist.
Please check out the site and let me know how you like it.
Update: We renamed the site to intotweets.com and are in the process of updating the design to reflect the new brand.
Yes, it looks like a stereotype. The first post in a blog usually says the blog launched. Same here, but I wanted to give you a bit of background on why I thought it’s important to lauch a blog.
- At PagePeeker, we develop out technology at a breakneck rate. From outside our office, this is not always visible, especially because we spend a lot of time working on the technology underlying the platform. So a blog will allow us to communicate better on what we do to improve our tech and in turn provide you with a better experience.
- Sometimes we launch new features. If they are important, we email about them. If not, or if you are not subscribed to our mailing list, there’s no other way to find out about them. A blog will provide us with a public space to share these updates with everyone.
- At exceptionally rare intervals, we make important announcements that affect many of our customers. When this happens, we will post them here on the blog and only link from the site main page. This will help keep our front page clean, while also allowing us to say what it is to be said in greater detail.
- Last, but not least, this will provide an open forum for our customers to leave comments and let us know how we did. We will, of course, moderate the comments, but will welcome and respond both to positive and negative feedback.
- Showcase our technology and some of the more challenging sites we can capture
Hope you enjoy reading.