To enable History Sync, navigate to the “Sync” pane of the Xmarks Settings dialog (Tools → Xmarks Settings → Sync) and click the checkbox next to History:

Do the same thing on two or more computers running Xmarks for Firefox, and those computers will automatically and silently start exchanging history information. Easy!

If you’re interested in understanding more deeply how it works and some of the design trade-offs we faced to make this happen, read on below the fold. And feel free to let us know how it works for you in the comments.

History Sync vs. Bookmark Sync

When we first started thinking about history sync, we naturally viewed it as being very similar to bookmark sync, and contemplated applying our existing Bookmark Sync Server to the task. Bookmarks and History being so similar in their structure, this seemed like an easy job. And our Bookmark Sync Server is battle-hardened and extremely reliable. Have you ever noticed that when you have a hammer every problem looks like a nail?

The problem crops up when you consider the volume of data generated by millions of users surfing and syncing their history. How many pages do you visit for each bookmark that you create? 100? 1000? Our Bookmark Sync Server is scalable, but as a small company, scaling a free service by a factor of 1000 is really out of reach for us. So we needed another approach.

Trying Not To Boil the Ocean

While there are lots of similarities between syncing bookmarks and syncing history, there are plenty of differences too. We started to explore those differences, looking for a more economical approach to the problem. The first observation is that users manage bookmarks, but the browser manages history. The browser takes all kinds of liberties with your history (e.g., creating new entries as you browse, purging old entries as they expire) that would not be acceptable when dealing with bookmarks. That yielded the first insight: since users don’t really “own” their history in the same way that they own their bookmarks, absolute fidelity is not critical for history. So we can probably deliver an acceptable experience without syncing everything in your history.

Next, the advent of the Awesome Bar in Firefox has really changed the way that users interact with browser history data. Power Users (who tend to be our biggest fans) learned soon after its introduction in Firefox 3 that the Awesome Bar allows them to get to their most frequently-visited sites just by typing a few letters into the address bar. The folks at Mozilla who designed the Awesome Bar spent time developing a clever algorithm to rank items in your history so that the most useful sites appear at the top of the drop-down list when you start typing.

The algorithm is driven by “Frecency” — a combination of the frequency with which you visit a site and the recency with which you’ve visited it. It tends to bubble up to the top of the drop-down list those sites that you have either visited a lot or have visited recently.

That leads neatly to the second insight: if you’re going to sync only a subset of the entire browser history, make sure that you pick the sites that are most frecent, because those are the sites that are most likely to appear in the Awesome Bar. A corollary here is that good history must be consistent with with frecency. That means that sync needs to accumulate visits to a site from all clients being synced so that frecency incorporates the sum total of all visits everywhere.

The final insight here is that, inasmuch as the browser purges expired items from your history as part of its regular operation, deletes don’t really need to be synced. That is, I don’t have to sync the deletion of old history items from one browser to another, because the other browser will probably purge those expired items on its own. That does neglect one use case that we decided wasn’t so important: syncing the manual deletion of an item from history. We figured that was a rarity, and in any event, users with foresight can gain the same protection by switching the browser into private browsing mode.

So what’s the result of this exploration? It’s all about the Awesome Bar, and feeding it the right data so that it provides good results when users type. It’s not about making sure that every change to history is synced to every other browser. We want to make sure that users can go back and forth between two computers they use regularly and get at sites that they visited recently on either computer, and also be able to set up a brand new computer and have the history get populated with the user’s most frequently visited sites.

The Solution

Given these requirements, our Bookmark Server looks like overkill: it provides complete data fidelity, it versions every change that a user makes so that they can do backup & restore, etc. For this problem, we need something lighter.

Happily, we’ve got another tool in our shed: to deploy tab sync, we developed a fast, lightweight server that is well-matched to the problem of syncing open tabs. It turns out that it’s rather well tuned for syncing history too. Here’s how it works:

• Periodically, the Xmarks addon queries Firefox’s local history database, looking for the most frecent urls, a set that changes constantly as you browse. Having determined the set of frecent urls, it then finds actual visits made on this browser to those urls. It builds a compact representation of this data and sends it to the server.
• Then it queries the server for history data from other browsers. If there is any, the addon downloads it and adds any as-yet unseen visit data to the local history database, making it appear as if you had made those visits on this browser.

That’s it. In actual deployment, the key variable is the size of the data being pushed from each browser to the server: the larger the data, the more history can be exported from one browser to another. But more data means more bandwidth and comptutation (for us and you). We’ve currently set the limit at 32KB, which allows for typically 200 or so urls. So far, that looks like a decent compromise, but we welcome feedback. Do you tend to find the things you’re looking for in the awesome bar?

In the coming weeks, look for history sync to make its way onto other platforms that we support.

Navigate and search your browser bookmarks:

Retrieve an open browser tab from any of your computers:

Switch between bookmarks & built-in browser without leaving the app:

When running on the iPad, version 1.2.7 gives you full screen browsing:

The Xmarks mobile app for iPhone and iPod touch is now available in Apple’s iTunes App Store at the less-than-a-latte price of $0.99.

It’s designed for Xmarks users that want to access all their desktop bookmarks and open tabs on their iDevice.

It works with all versions of the iPhone & iPod touch, with iPad support coming soon.

Key features include:

• works with Xmarks for IE, Firefox, Chrome, and Mac Safari
• access all bookmark folders including your bookmarks toolbar
• bookmark search
• open remote tabs across all your computers & browsers
• built-in browser to view your bookmarks without leaving the app

UPDATE: Version 1.25 is NOW AVAILABLE in the iTunes App Store! This version adds a more streamlined user interface and a fix for the iPad bug in the previous version.

Coming Soon to Xmarks for iPhone:

• improved user interface
• ability to add and edit bookmarks for 2-way sync
• full screen iPad support
• and lots more!

Remember that even without Xmarks for iPhone (or an iPhone for that matter), you can point any smartphone to my.xmarks.com and get mobile access to your bookmarks and open tabs.

Since we first launched Xmarks for Chrome last August, you’ve been asking us when we’re going to add password sync.  Does today work for you?

Xmarks now synchronizes your bookmarks, passwords, and open tabs across both Firefox and Chrome! 

UPDATE: We’ve released a new version of Xmarks for Chrome that includes (as a default-off option) password sync. You can find it here in the Google Extensions Gallery.

Just like our password sync in Firefox, password sync in Chrome is 100% optional and is turned off by default. Your passwords are secured by encrypting them before they leave your computer with a PIN that only you know.  (Learn more about how we keep your password information safe.)

Unlike Firefox, Chrome’s built-in password manager is not available to extension developers; so instead we have created our own Xmarks password manager that you can access from the Xmarks settings. And if you need any more encouragement to give it a try, remember we recently released Restore for passwords, so you can go back again if anything goes wrong (which it won’t!).

Please give it a try and let us know how it goes for you in the comments here or on our support forum.

Congratulations, Mozilla, and thank you, Xmarks users!

For most people, this should be a fairly uneventful update. But the new version does contain a raft of changes, and we wanted to highlight some of the more significant ones.

Update: If you’re having problems logging in with this version and you have also installed NoScript, make sure that you have whitelisted xmarks.com.

• Password Restore: We’ve supported bookmark restore for quite some time now, a feature that our users regularly tell us has saved their bacon after a hard-drive failure or other corruption. We now also support password restore, which we think will also prove indispensable if you experience an unfortunate data loss. The feature itself lives in My.Xmarks, and it works with this version of the Xmarks addon. You can find it in the Tools menu of My.Xmarks.
• Authentication: The way we handled authentication previously caused inconvenience to users who either enable the Firefox Master Password or don’t let Firefox store passwords. If you’re one of these users, you know that automatic sync would trigger a password prompt the first time Xmarks synced after a browser restart. That’s no longer the case. Once you log in, you should remain logged in; just remember to check the “Remember Me” box.
• Suspend: For most users, Xmarks operates transparently in the background. For users with huge datasets and older computers with limited memory and CPU, that’s not always the case. We’ve recently made improvements in Xmarks’ efficiency, but now there’s another option: if you log out (press the Log Out button the first page of the settings dialog), Xmarks will suspend automatic sync until you log in again or restart your browser. This can be handy when you’re working on something else and don’t want Xmarks to get in your way.
• Detect Offline: If you’re on a laptop with a wireless network connection, you know that your connection comes and goes depending on your location, or whether you’re putting your laptop to sleep or waking it up. Previously, if your network was unavailable when Xmarks decided it was time to sync, you’d see an error in the status bar. Now, Xmarks will defer automatic sync requests until the network becomes available again.
• Sync on Shutdown: Xmarks has always provided the option to sync your data when the browser is shutting down to ensure that any recent changes you’ve made are available to other browsers and computers that you use. Previously, if Xmarks had any changes to sync, it would sync every datatype that you had enabled: bookmarks, passwords, and tabs. Now it syncs only what’s necessary, making shutdown substantially faster.
• And more! There is a handful of other minor changes and improvements, ranging from password-sync bug fixes to the ability to cancel an in-progress automatic sync. We hope you enjoy the new version and (if you celebrate it) have a great Independence Day.

We almost forgot to post this… A few weeks ago one of our favorite sites, Lifehacker.com, ran a user poll to find out who to crown as “Best Bookmark Management Tool”. Thanks to lots of loyal Xmarks users who voted, we crushed it and took the top spot with almost 50% of the total votes cast!

More details and comments at the full Lifehacker article.

We were delighted to see one of our Xmarks users post this image to Flickr: three browser favicons, including Xmarks, created out of Hama Beads!

Thanks and photo credit to lwordish2010. It’s great to know our logo holds up at hama-bead-resolution ;-)

To check it out, open your browser and log into your account at My Xmarks.com. If you’re using an iPhone, you’ll see something that looks like this:

Click the Open Tabs button to see a list of your computers, browsers, and their open tabs. Click on a tab and presto! It opens on your phone.

If you’re using a browser on a desktop or laptop computer, you’ll find Open Tabs in the Tools menu of My Xmarks.com:

What do you think? Let us know in the comments.

The update will be rolled out to all of our Xmarks for Chrome and IE users this week. (If you can’t wait, get the Chrome version here, or the IE version here.) We hope you enjoy it—let us know how it’s working for you!