WinFS

WinFS Beta 1 logo

WinFS (short for Windows File System or Windows Future Storage) was a canceled storage subsystem that allowed users to search for items based on their metadata, regardless of the item's type of file and the application in which it was created. This in turn would make searching for content independent of format easier.

WinFS was billed as one of the pillars of Windows Longhorn. However, Microsoft announced on 27 August 2004 that WinFS would be in beta testing when Longhorn becomes available, and would be delivered after the Longhorn release. On 29 August 2005, Microsoft (through Vijau Bangaru, a Program Manager on the WinFS product team) announced the release of WinFS Beta 1 for Windows XP Service Pack 2. Regular beta and CTP releases were planned.

On 23 June 2006, Microsoft (through Quentin Clark, who ran the Program Management Team for WinFS) announced that there would no longer be a separate WinFS release. Instead, some of its parts were included in ADO.NET and Microsoft SQL Server.

WinFS was based on SQL Server 2005, ran on top of NTFS, and used metadata of files to improve organizing, searching and sharing of information. It also stored structured data such as contacts and calendars. WinFS replaced the traditional file system hierarchy of files and folders with items (either based on files or not, such as contacts and e-mails) and associations (relationships between items).

Development

WinFS is ultimately traceable to the Object File System (OFS), an integrated store, a data store, for storing document files, spreadsheets, multimedia files and other information which Microsoft planned to be part of Microsoft Cairo. The idea behind OFS "was to make sense out of all the bytes of data in the filesystem in a human readable and searchable way, organized as a collection of objects with properties all part of a strongly typed schema." OFS was a relational file system, "which would help manage local and distributed objects using an active storage paradigm" and "would have removed the need to understand where data resided", both locally and remotely. One build of Cairo, 1175.1, allowed OFS partitions (see screenshot). However, it is not clear if the features described for OFS are present. Instead, OFS, as implemented in build 1175.1, appeared to be a predecessor of the Content Indexer (now the Indexing Service) and Active Directory.

• 

  Screenshot of Microsoft Cairo/4.0.1175.1. Note that local disk D:\ is under an OFS partition

• 

  Info Center from Windows 95 (Chicago) build 58s, an application that allowed emails to appear inside Windows Explorer. The Communication History seen in Longhorn builds performed the same function. WinFS stored the emails synced from Outlook Express. See images below.

Cairo fell apart in the mid-1990s. On March 1996, Microsoft said that work on OFS for a future release of Windows NT had been abandoned in favor of a file system based on its Exchange Server. The file system work was abandoned because of complexity, market forces and internal bickering.

OFS merged into Storage+, which Paul Thurrott wrote, Microsoft would turn its attention to after Windows 2000 ships. Storage+ would replace the NTFS file system and with a relational, object-oriented file system based on SQL Server 8.0, turning the file system into a relational database, speeding searches and providing a higher level of granularity for those searches Storage+ later became the Relational File System, which Microsoft intended to but failed to ship with SQL Server 2000.

Storage+ (at this point "a new SQL Server .NET-based file system") was then slated for Blackcomb, but by 2002, Thurrott verified that it would ship with Longhorn. CNET News.com reported, "We're going to have to redo the Windows shell; we're going to have to redo Office, and Outlook particularly, to take advantage" of the new data store, Microsoft CEO Steve Ballmer said in a recent interview with CNET News.com. "We're working hard on it. It's tough stuff." Jim Allchin, the group vice president in charge of Windows development, said, "bringing the plan back now makes sense because new technologies such as XML (Extensible Markup Language) will make it much easier to put in place. XML is already a standard for exchanging information between programs and a cornerstone of Microsoft's Web services effort, which is still under development. Longhorn and the new data store are the "next frontier" of software design, Allchin said. In addition, Microsoft has already developed the database technology it needs for a new file system. A future release of its SQL Server database, code-named Yukon, is being designed to store and manage both standard business data, such as columns of numbers and letters, and unstructured data, such as images."

By 2003, Storage+ was called WinFS, would include technology from SQL Server "Yukon" (SQL Server 2005), and would not be a file system but a service on top of NTFS.

During PDC 2003, Bill Gates described WinFS as a "Holy Grail":

WinFS, this is unified storage. Some of you here have heard me talk about unified storage for more than a decade. The idea of taking the XML flexibility database technology and getting it into the file system, that's been a Holy Grail for me for quite some time, and here it is thank goodness. We've got the powerful systems to be able to do this. Thank goodness, we have the evolution around XML and user interface capabilities, so that this this can all come together.

— Bill Gates, PDC 2003 Keynote with Bill Gates,

Bill Gates had earlier described OFS as a "Holy Grail" in his PDC 1996 keynote.

In his PDC 2003 presentation, Quentin Clark, director of program management, WinFS, explained:

WinFS is...

• All end-to-end data lives in Longhorn
• New user experience in the Longhorn shell
• A trustworthy place to store data
• Data model built on relational database technology
• File system capabilities built on NTFS
• Everyday info - domain specific schemas - Longhorn will come with set of schemas Services that make data interactive

WinFS is not...

• Abandoning legacy applications
• Forcing the world to rewrite everything
• Complete yet
  • PDC is good preview of model, feedback welcome
  • Fundamental model complete for Beta 1
  • Platform and UE complete in Beta 2
• Just a database - MS has set of relational assets, WinFS is pulling techs from NTFS, SP, and CLR too

— Paul Thurrott, Keith Furman, Live from PDC 2003: Day 2, Tuesday,

This WinFS schema diagram dates from PDC 2003.

On 19 April 2004, BusinessWeek reported that Microsoft would scale back WinFS to provide document indexing only on the local system and not over a network.

On 13 May 2004, it was reported that Bob Muglia, the senior vice president in charge of Windows server development told CNET News.com, "some of the functionality of WinFS and some of the scenarios may be limited in terms of what it can do. I don't know that we will have all of the scale to the level where we would like to have it, so that you could use it for very high-volume enterprise servers," and, ""So it could be that for example, you could use WinFS as a server for collaboration in workgroups. But if you want to support hundreds of users, that may wait for the update release."

On 27 August 2004, Microsoft announced that WinFS would be in beta testing when Longhorn becomes available, and would be delivered after the Longhorn release. This meant that WinFS would be removed from Longhorn and would ship at a later date. Thurott wrote, "The removal of WinFS, however, had wrecked havoc with a number of product groups. The next version of Outlook Express (now called Windows Mail) and Contacts were originally being built around WinFS and had to be rearchitected. Internet Explorer was using it for storing history and Favorites. Even Office 12 was building on WinFS. All of that work--and more---needed to be rethought."

A report from 10 December 2004 quoted Muglia as saying that WinFS would neither be in Longhorn nor in Longhorn Server. Muglia also said, "Our mistake, frankly--and it was a mistake--was to go out and talk about it before we really had clarity as to how we would be able to deliver it and all the complexities there." He then described WinFS: "This isn't a relational database"; "This is a brand-new data model, and it satisfies a whole class of applications that frankly have been unsatisfied from a data model perspective since the beginning of history. We've been working on things like this for a long time."

On 29 August 2005, Microsoft quietly made WinFS Beta 1 available to MSDN subscribers, but announced it officially through the WinFS Team Blog. Beta 1 required Windows XP Service Pack 2, and .NET Framework 2.0.50215.322. The WinFS API was included in the System.Storage namespace.

During PDC 2005, Microsoft released a video presentation, named "IWish", showing how applications would advantage of a unified type system. The concepts shown in the video ranged from applications using the relationships of items to dynamically offer filtering options to applications grouping multiple related data types and rendering them in a unified presentation. Steve De Mar, the Product Designer for the WinFS team, explained the goals of IWish:

Our goal was to peak the curiosity of you, the developers, to get you thinking about what could really be built on WinFS. We wanted you to see how WinFS enables a totally new class of user experiences. The UIs in the video are all based on some of the great prototype applications that have already been developed for the WinFS platform. Although we cast them in a more futuristic visual “theme” to fit the style of the video, the functionality we showed exists in the prototypes that either we’ve developed in house or we’ve seen others develop on WinFS. A number of these were demo’d at the PDC, and we've more on the way which you should be hearing about shortly.

We hope the video inspires you to bring to life cool new innovative applications for WinFS!

In addition, during PDC 2005, Shishir Mehrota of WinFS Product Planning gave an overview of WinFS, Program Manager Ramesh Nagarajan gave a presentation on "“WinFS” Future Directions: Building Data-Centric Applications Using Windows Presentation Foundation “Avalon” and Windows Forms", (for which he built a simple Windows Presentation Foundation application that showed off how a richer document viewing experience could be built on top of WinFS) and Samuel Drucker, Development Manager, gave a presentation on WinFS and ADO.NET (on how ADO.NET is evolving to incorporate some of the WinFS aspects) and WinFS Programming with LINQ.

Beta 1 Refresh was released to MSDN subscribers on 1 December 2005 to be compatible with the final release of .NET Framework 2.0.

Four WinFS sessions were planned for Tech Ed 2006. Roger Lueder gave a talk on "Building on WinFS: Developing a Great WinFS App from Scratch".

Beta 2 was planned for some time later in 2006, and would have been integrated with Windows Desktop Search, so that the latter could display both WinFS and non-WinFS files.

But on 23 June 2006, Quentin Clark announced through the WinFS Team Blog, "With most of our effort now working towards productizing mature aspects of the WinFS project into SQL and ADO.NET, we do not need to deliver a separate WinFS offering."

Many of the principal features Microsoft intended to provide with WinFS included a pane for metadata property editing, breadcrumb-based property navigation, filtering or stacking items over properties, incremental search, and saved searches; these features were incorporated in Windows Vista. Query composition, a WinFS feature that allowed users to perform additional searches that reuse the results of a previous query, was later incorporated in Windows Vista.

Examples of uses of the technology are the object-relational mapping components into ADO.NET Entity Framework; support for unstructured data, adminless mode of operation, support for file system objects via the FILESTREAM data type, and hierarchical data in SQL Server 2008, then codenamed Katmai, as well as integration with Win32 APIs and Windows Shell and support for traversal of hierarchies by traversing relationships into later releases of Microsoft SQL Server and the synchronization components into Microsoft Sync Framework.

In 2013, Bill Gates cited WinFS as his greatest disappointment at Microsoft:

We had a rich database as the client/cloud store that was part of a Windows release that was before its time. This is an idea that will [re-emerge] since your cloud store will be rich with schema rather than just a bunch of files and the client will be a partial replica of it with rich schema understanding.

Limitations of traditional file systems

Traditional file systems, including NTFS which is used in modern versions of Microsoft Windows, store data only as a stream of bytes packed in a file. The Microsoft WinFS SDK Beta 1 Documentation explained the limitations of this approach:

In a file system, data is stored as one or more byte streams packed in a file. This flat data structure does not make the information easily identifiable. The only viable way to identify it is to use the file name and its directory path, that is, a folder hierarchy. Although a file may contain many kinds of information, it is the smallest unit of data that can be moved around and operated on. For example, to read a paragraph of a document, you need to retrieve the entire file and open it in a document viewer.

As Richard Grimes explained:

The perennial problem with data is that you always have too much of it and it is always too difficult to find what you want. This problem will get far worse in the near future when typical machines will have hard disks measured in terabytes. The bigger the hard disk, the more data you will want to store, and the more difficult it will be to find a particular item. Finding a piece of data depends on identifying the specific item, which means that the item has to be categorized.

For example, imagine that you have several thousand digital photographs. Typically, your camera and desktop software will catalog this data by naming the photos in sequence and maybe place these in a folder according to the date that the photos were taken. Searching for a particular photo in this scheme involves remembering the date and photo number. If you do not know either of these things, you will have to resort to looking through thumbnails of the photos. Of course, if you have the patience, you could name each photograph as you download it to your desktop. A coarse-grained system, like the file and folder system in Win32®, is not flexible enough to allow you to store all the data you need.

Desktop software may fill this gap, allowing you to build a catalog that stores other descriptive properties like camera settings, light condition, and a general description of each picture. Such software, of course, is an implementation of a database, but the important point is that it acts as a supplement to the file system. You could argue that these descriptions of your photos—metadata—are actually part of the photograph's object. Digital photographs hold other information within the image file's EXIF (Exchangeable Image File Format) header. This header contains information obtained when the photo was taken. To include this metadata in a search would require a query engine to extract this information from each candidate photo and compare it with the query condition. A sophisticated photograph organizer could extract the EXIF information when the photograph is transferred to the desktop and include it as additional metadata in the database.

Desktop search allowed a user to see different types of data from different sources. However, these data still resided on different sources, different places.

Traditional file systems also did not allow for storing files in multiple places without making duplicates. Hal Berenson, former General Manager of Integrated Storage (which would become WinFS after he left Microsoft), gave an example:

For example, you might want an album with pictures of Aunt Jean. But you also want the pictures of Aunt Jean at the wedding to be in the wedding album. You also want to share about 50 of the 500 photos you took (and make sure you don’t share any of the hot tub pictures). How do you do that without copying the pictures to a separate share location? Maybe you want to organize photos from all visits to Disney World together, but also keep them together by broader trip.

A solution is to put an external metadata later on top of the file system, and to use an application or library to interact with the photos instead of interacting directly with the file system. Berenson noted that this causes problems as applications typically won’t understand the layer and operate just on the file system underneath it. This can make functionality that the layer purports to provide unreliable (e.g., when the application changes something about the photo which is not accurately propagated back into the external metadata store). And with photos now stored in a data type-specific layer, it is ever more difficult to implement scenarios or applications in which photos are but one data type.

Berenson then illustrated the limitations of traditional file systems and the benefits of integrated storage by using the processing of claims against an insurance company as a scenario:

Let me cross over into the enterprise app space and talk about an Insurance Claims Processing scenario. Claims processing is interesting for a number of reasons, they key one being that it was one of the first enterprise applications to really embrace the notion of multiple data types. When you file a claim, for a car accident for example, it goes into a traditional transactional database system. But each claim has an associated set of artifacts such as photos of the accident scene, the police report, photos taken by the insurance adjuster, photos taken at the repair shop, witness statements, etc. that don’t neatly fit into the classic transactional database. Yes you can store these artifacts in a database BLOB, but then they lose all semantics. Not only that, you have to copy them out of the database into the file system so that applications that only know how to deal with the filesystem (e.g., Photoshop) can work against them. And copy them back. That creates enormous workflow difficulties, introduces data integrity problems, and prevents use of functionality that was embedded in the photos storage application.

The claims processing scenario is one that demonstrates where the name integrated storage came from. What you really want is for the same store that holds your transactional structured data about a claim to hold the non-transactional semi-structured artifacts, and not just as blobs. You want the semi-structured artifacts to expose their metadata and semantics to the application, or applications, built on that store. As soon as you do that the ability to create richer apps, and/or use the data in complex information worker scenarios, climbs dramatically.

Rather than just using the photos as part of processing a specific claim they now become usable artifacts for risk analysis, fraud analysis, highway planning, or any number of other applications. Data mining applications could run against them seeking patterns that weren’t captured in the transactional data. Indeed all kinds of linkages could be made amongst the photos, police reports, etc. that just aren’t possible from the transactional data alone.

Lastly, Berenson noted that there is no way to back up and restore related structured (i.e., typical database data), semi-structured/unstructured data (e.g., a photo or document), and unstructured data, without a storage system that can be the primary store.

Grimes pointed out that one of the monumental problems organizations face is aggregating information that is stored in disparate formats. Knowledge workers have long wanted to be able to search for content independent of format. Sean Grimaldi of the WinFS team called the existence of many formats with different expectations and behaviors a "divide-and-conquer approach" in dealing with data. Grimaldi extensively discussed the negative effects of this approach and summarizes:

The data is divided into file, database, and other data. The negatives are that programmers have to recreate data types, bridge between data types, and rewrite behaviors repeatedly. If this seems normal to you, it may be because you have operated in this fractured data environment for years and have come to expect nothing more.

The Microsoft WinFS SDK Beta 1 Documentation described the existence of proprietary file formats as "separate islands":

Proprietary file formats tend to aggregate data into separate islands, creating barriers for data sharing between applications. For example, an address book created by an e-mail system, such as Microsoft Outlook, is not directly usable by an appointment-setting application unless the latter adopts the same file format or knows how to translate the format from one to another.

Shawn Wildermuth pointed out what this system means for developers:

Data integration solutions must first try to make sense of all the different places applications can store information. Let's look at a typical example:

I am writing an application, I need access to the user's Contact List, Documents folder, and Music Folder. To access each of these data stores, I have to deal with three different APIs to get that data (MAPI to access Contacts, System.IO for the Documents folder, and the Windows Media Player SDK to find all the music for a user). It would be better if we had a consistent API for users' data.

But that is not the worst part of the problem. If my application wants to tie its data and these external pieces together, it will need code that not only establishes the relationship between the objects, but deals with changes to that data and handles data synchronization problems such as orphaned data.

Using common file formats is a workaround to this problem but not a universal solution; there is no guarantee that all applications will use the format. Other solutions used by corporations are database systems and XML and XML web services, as Thomas Rizzo, director in the Microsoft SQL Server group, explained:

In the beginning, computers could not handle the complexities of modeling and automating many business processes. However, with faster computers, better programming languages, and better data technologies, automation of business processes became a way for corporations to streamline their operations, especially with the advent of enterprise resource planning (ERP), customer relationship management (CRM), sales force automation (SFA), and other types of enterprise applications. With these new types of applications, data suddenly became more complex—but at the same time, data had better structure that was more useful for businesses. Rather than storing opaque binary data or simplistic data models, database systems could relate different types of complex data together. For example, ERP systems understand and can model the complexities of general ledgers, human resources, and sales systems. Since standardized schemas described the data in the system, corporations could ask interesting questions about their business and query their systems for that information.

However, working across systems is still a problem in the industry. Not only is it hard to integrate systems within a corporation, but integrating systems across corporations is even harder. This has sparked the growth of enterprise application integration (EAI) vendors who help customers navigate the sea of schematized data across heterogeneous systems. Recognizing this integration problem, the industry has looked to XML and XML Web services as a way to help companies work across multiple systems and multiple organizations. These newer standards help solve data integration problems and are helping to democratize data throughout and across organizations.

Traditional file systems also provide only a single way of organizing the files, namely via directories and file names. Vijay Bangaru of the WinFS team illustrated the problems with this:

If you’re like me, you probably have tons of digital pictures on your computer. My photos are currently organized in a simple hierarchy. In “My Pictures”, I have a bunch of folders that are named something like, “2004-11 Thanksgiving”. Within this folder are all my photos from that holiday. I’m sure you can already see the limitations in this structure.

My latest challenge in actually using my pictures came the other day. My grandparents wanted me to send them some pictures of my fiancée and me. Wouldn’t it be really cool if I could just tell my computer, “Show me all the pictures that have Crissy and me in them”. Instead, I had to scan through my folders to see what events we were both at and copy all the pictures with both of us in them to a temp folder. Then I needed to go through the temp folder to pick which ones to send. Of course, I didn’t remember to delete the temp folder; so it’s eating up drive space. Good thing hard drives are so cheap nowadays.

A traditional file system can retrieve and search data based only on the directory path:

If the user knows the directory path to the file, searching is simple. But if she does not know where the file is, she must walk down from the root directory of a file system—for example, by selecting Search from the Start menu—to traverse the available directory paths until she finds the file. When there are many files on a computer, the search can take a long time. In the NTFS file system and other traditional file systems, the directory path is more or less the only way to identify a file. But users tend to think of data more naturally in terms of its types and attributes—that is, the structure or shape of the data: a document or an e-mail message, its title or subject, the author or sender. How the data is stored is generally of little or no interest. Therefore, reliance on paths is an inherent limitation of the tree-based structure of traditional file systems.

A user may put a file into multiple directories, but this wastes disk space. Also, when a copy is changed, the other copies no longer represent the same data as the changed copy. Hard links that can be created to represent a file by multiple paths are also difficult to manage: links will break when a file is renamed.

Bill Gates summarized all these issues in his keynote during PDC 2003:

Users want to manage information. We've had the information in silos. Mail is separate than, the files is separate from the web pages, and the information has not been structured. All the operating system knows about files is that they're binary streams. That's it. You know, something like the address book is at the application level, not at the platform level. And so if you want to share address book across applications, you can't even do that. The idea of rich searching, the search command is different in every application. So learning how to search mail, navigate mail, is different than learning that for files. When you get attachment sent to you, storing those, understanding the different versions you might be getting, It's very complex, and so just dealing with information has not been very rich. We don't have standard schemas for the concepts. We had the clipboard, but we haven't gone beyond the clipboard to structure the information, to make replication, and searching just a very straightforward thing that you expect out of the system. So in the next wave, we've got to have those rich views, got to have content indexing, got to unify the store. We've got to have the schemas, allow you to classify things, so that things even appear to organize without you getting involved. We see a little bit of this in the Outlook client where you have search folders, so that if you want to see all your mail from one person, you just click on that folder. You don't have to drop something in there because it's, it's essentially a standing query, and so information can be in many locations. It's just a matter of defining the criteria, not having to move it around. The ability to intermix files and mail and notes into one hierarchy that's organized the way that you want it to be.

WinFS as solution

The core concept of WinFS was that by storing data in a cohesive data store, not just a file system, it would have allowed systems and users to store rich metadata about a myriad of objects and to relate these objects. Compared to a file in a traditional file system, a "WinFS" item would be the smallest unit of data that could be moved around and operated on by different users or applications.

The Microsoft WinFS SDK Beta 1 Documentation explained that WinFS would have provided comprehensive object-relational mappings to let users view and work with their data as an item on a Windows computer. With items, a user or an application would have found it more logical or natural to view and use data. This should have made organizing and querying data, especially complex data, less burdensome. The documentation provided as an example:

For example, consider an e-mail message containing two photos as attachments. In the tabular form, this is most likely modeled as three rows in two tables. One table stores all the e-mail messages and the other stores all the photos. To get the message, a user or application must query two tables and perform the so-called UNION operations on the tables. In the itemized form, the attachment is part of the item. Querying the e-mail message item will return the attached photos as well.

WinFS would have served all kinds of data, whether structured (as in tables/rows or items), semi-structured (as in an XML file), or unstructured (as in a flat byte stream).

The Microsoft WinFS SDK Beta 1 Documentation and the IWish video summarized the benefits of WinFS integration into four points: Unify, Organize, Explore, and Innovate.

Unify

WinFS would have allowed data sharing between applications, as Grimaldi explained:

Because applications can build on WinFS-defined data types, applications can share data—assuming the correct permissions. This is good for you as a programmer, as well as for the end user of your application.

As a programmer, you can reuse data the end user already has in the WinFS store. For example, assume your application presents advertisements to a user of events they could attend, like a bike race or a rodeo.

Before presenting an event to the user, your application can check the user's calendar and display the event differently if the user already has a meeting scheduled that would conflict with the event, such as making the border of the invitation red.

You can also populate data into the WinFS store to be used by other programs. For example, if the end user accepts the invitation to the bicycle race, their personal information manager (PIM) application, such as their e-mail client, could show that date as reserved in their calendar. This also gives the end user a rich user experience in both applications.

WinFS would have placed data from multiple applications into one central store. Bangaru showed the advantages of this:

With WinFS, I can move all my contact information into one central store. My applications can interact with this information and I can synchronize it wherever I want to. The obvious benefit is that I can go to one place reliably for the most up to date information. Now, I could easily add whoever I wanted to an Evite for a Halloween party. No more need for typing in email addresses or importing all my contacts (into yet another silo). When I order flowers for mom on her birthday, it’s just one click for me to fill out the information for the recipient.

In addition to being a central data store, WinFS can be your synchronization hub. I can sync between WinFS Stores and I can sync changes back and forth with legacy stores (like Outlook, PDAs, and cell phones). Now, I can have my latest contact info on my phone, make changes on the road, and sync that information back to WinFS. This, of course, enables some great offline experiences. When I’m doing work on vacation, I don’t need to worry about manually synchronizing my data with my desktop machine once I’m home. Now all my applications will behave like Outlook 11’s awesome offline mode.

Of course, this applies to all data; not just contacts. Now, all my calendar apps will show my Evite invites. My photos are now associated with contacts and events, so I can easily say, “Mail these photos to everyone that was at the event”. When you start adding email, RSS feeds, documents, music and everything else to this, it can get pretty exciting.

Organize

Bangaru illustrated how data could be related, and how related data could be shown:

With WinFS, I can relate my contacts to my photos, so I can easily query for all the pictures that are related to a set a contacts. A photo app built on WinFS could display all the pictures with Crissy and me in them. Then, I could select the good ones to email or print. If you’ve been playing with the Beta 1 bits, you know how easy it is to write an app with this behavior.

Plus, I can now easily check to see what photos I’ve already sent, so I don’t send the same ones again. How? Tell the computer to “show me the photos with both of us that I’ve sent to my grandparents.” All these containment relationships and associations would already be in my store.

Again, this applies to much more than pictures. I can associate a document with a meeting, or music and contacts to a party invite. And these aren’t keywords added to a file, these are associations between real items. All of these organization constructs are just waiting to be exploited by a WinFS app.

Also, all these types are extensible. So for my wedding guest list, I can extend the Contact type to have properties for “Sent ‘Save the Date’ card”, “Sent invitation”, “RSVP Received”, “# of Guests”, “Vegetarian meal?” and so on. That could make things significantly easier. Good thing I can do this with the Beta bits. If we had to wait for WinFS RTM to plan the wedding, I think Crissy would make sure that I wouldn’t be able to blog anymore. :-)

Explore

WinFS natively recognized different types of data by defining common item types, such as audio record, meeting, contact, person, document, photo, message, video clip. This would have solved the problem of querying the following data:

You could write code that shows a picture of a person, his role, and his contact information for everyone attending a given meeting and every document that they contributed to that you have not already read, and on a personal note, if they have already given you some money for the charity bicycle race you are participating in next month. Today, I have all this data available to my computer, but no feasible way to query it without WinFS. I plainly can't get at a lot of my data.

WinFS would have allowed queries such as "Show me all email from people I am meeting with this week," that NTFS could not.

Bangaru explained how one of the mockup applications showed in the IWish video would use WinFS:

WinFS gives you the ability to use relationships to find and display your data. In the PDC video, we showed an app that takes a range of dates and displays the location for your meetings on the map. As it draws the best route to drive between all your meetings, it adds icons that represent your contacts and important customers at their location on the map. Now, as you visualize your schedule, you can see who is close by and set up lunch.

Let’s walk through how WinFS made this possible. The app is able to query for all your appointments in a certain date range. It displays each appointment in the proper place on the map by using location of the meeting. Today it would be tough for a map application to do this, because your appointments are in a different application or maybe even more than one application. Assuming you can access the meetings programmatically, accessing the location may be hard. Your calendar app probably just has a field called “Location” that can contain an arbitrary string and would be tough to parse correctly. With WinFS, all your appointment information is in a single store with an API that let’s any application access the data uniformly.

Next, the application finds contacts and customers with high priority issues that are located close to where you will be and displays them on the map, by using the address of the contact. If you select a contact, the app can display some information for the person (a photo and contact info for example). From there, you can send them a mail to setup a lunch while you’re in town. Again this would be tough to do today; your contact information is probably in a silo that your map application can’t get to. Plus, chances are, your customers with high priority issues are tracked in a completely different application silo.

Innovate

Bangaru explained how WinFS would have done away with proprietary file formats:

WinFS is built on a true relational store that all applications can leverage. Today, tons of popular apps lock their data into proprietary stores. This includes photo apps like Photoshop album, music apps like WMP and iTunes, financial apps like Quicken and Money, email apps like Outlook, and so on. By using WinFS, developers can concentrate on building rich applications without worrying about creating a storage silo. After all, WinFS can give applications a place for all their structured, semi-structured, and unstructured data.

In addition, WinFS will provide services like synchronization, notifications, rules, and backup/restore. The services provided by WinFS lead to a zero-admin and extremely reliable experience.

Drucker explained how WinFS addressed lock-in:

Interviewer: "So, is Microsoft getting rid of lock-in? I mean, is this the uh, are we, are we docking in a new pool here?" (laughs)

Drucker: "Well, those sound very buzzy." (laughs)

Interviewer: "Probably"

Drucker: But I will say this. WinFS is a persistence platform because of the schematization, because the programming model enables applications to share data on behalf of the user in a way that just isn't even feasible to do now. Even if, even if applications wanted to, you really couldn't do it. I mean you can't write outlook express today as like a file folder of MSGs. It's just not fast. It's not practical. Users will hate it."

Interviewer: "Now that for, for an application developer who is used to lock-in and used to "I control my world", that sounds scary, doesn't it?"

Drucker: "I think it is very scary and I think we get that feedback, we've gotten some of that feedback from ISVs. But I think the the upside, right, the world where you can share information with other applications is so powerful that even people who enjoy a tremendous amount of lock-in in their applications today think about 'Oh what would it mean if I could really share data with the other applications the user uses?' And that promise, that potential of greater interoperability far outweighs the benefits they're getting from lock-in today."

Synchronization is explained under Data sharing. Rules is explained under Rules.

Bangaru explained two levels of backup that WinFS would have included. Item level backup would have allowed a user to revert a change to an item. Store level backup would have covered the situation of a broken hard drive and a user needing all his files back.

As part of Longhorn

Concepts

"Since music place is not a physical space we can promote many virtual views. One we promote here is "most recently added" grouped view where cds users have copied recently are populated." (Source: Heesung Koo)

Hillel Cooperman explained in PDC 2003 in his demo of a build 4050 how WinFS would work in Longhorn:

But the best part about this is not that we have this functionality built in Windows. The best part, like everything I want to show you today, is that this is part of the platform. This is essentially the evolution of My Documents, but a couple of things are different. First of all, we're showing almost 1100 items in this view, which frankly, given today's uh experience, it would be a little uhh not useful. Um and in fact, these items are not sitting in one folder. They're located across the system, and this view is actually a dynamic view that's generated for you, uh right in the system, and of course available as part of the platform. So let's do a simple thing uh, that you think would be simple today, but, but WinFS is going to provide built-in search facilities. So I have 1,100 items, I want to fall, find all the items that have somebody do with Longhorn. So as I type, I want you to watch the the number right here, 1095, go down to 30. See how fast that happens. By the time I'm done typing, we're down from 1100 to 30 items right there in the system. Now built-in search capabilities, this is something you're gonna expect.

But let's talk about the kind of self-organization that Bill talked about in WinFS. For example, let's say we want to take these 1,100 items and organize them by a property, a property that's built into the schema. Um, so right here, we have "Author". We click on "Author" and we could actually filter down the 1100 based on who the author is. Or we can do this new thing called "Stack" and when I click on Stack, it's actually drawing a new view that shows all these items uh in these stacks by who wrote them. And in fact, if you look carefully at the stacks, they have different heights depending on how many items are represented in them. And because they're not physical locations, one item can show up in more than one stack. And these are not folders I went, created. This is dynamic view being built for me on the fly based on the rich structured metadata that's sitting in WinFS on these items. So let's double click on "Bill". Now, the properties that are built in to the schema are well and good, but of course you're gonna want to add your own. And it's not just users and IT professionals but of course developers. They're gonna go extend these schema uh, in any way they want. So let's actually add a property right now, let's add project to our view. I click right here on "Project" and I can stack by project. And in fact, I can go down here and see here's all the stacks. And these five items down here aren't attached to any project.

— Longhorn.ms, Hillel Demoes Longhorn build 4050 (Extended cut),

WinFS was intended to have the following user interface (UI) features in Longhorn:

• A breadcrumb bar for navigation
• A preview area
• Commands (based on FolderType and folder sniffing)
• Filters and filter controls
• A list view area (for thumbnails and grouping)
• A user's list of storage favorites

The storage model has some other elements which are discussed below — for example:

• A scope (for example, c:\foo + d:\bar)
• A specific query and filters (for example, select * from Media.Music where Artist = Madonna & year = 1984)
• An associated "drag target" (a physical folder, which is optional)

• 

  All Albums: "Expose five pinned pivot in music library which are all tracks, all albums, all artists, all playlists, and most recently added. There are more auto preset using usage pattern under more auto-search." (Source: Heesung Koo)

• 

  All Artists

• 

  Filter 1: "Now in LH build you can see that it shows all sub catagories under genre, year, size and duration. Even though the catagories are different here. In music UX team will decide what we think to be filter lists here"; "The default will be albums and artist, but the filter area will be MRU, so that it will be changed whatever you used recently." (Source: Heesung Koo)

• 

  Filter 2

• 

  Virtual folder - all songs: "Users click all songs pivot LH will return user with flat list of all of their songs with ability to sort the collection easily." (Source: Heesung Koo)

• 

  Virtual folder - Most frequent played

• 

  Virtual folder - all artists 1: "All artist view is optimized around using groupings to contain all albums users have on the artist"; "When users click the group title, it will be selected all albums under artist." (Source: Heesung Koo)

• 

  Virtual folder - all artists 2: "We want to show the songs that are in the albums as well under artist"; "If users want to select all songs then they can select either album title or album artist. Also they can drag and drop albums or songs in the player"; "Users can select the group header to select all albums and add these albums to currently playing play list." (Source: Heesung Koo)

• 

  Virtual folder - all artists 3: "Users can also choose to use the search field to essentially query on their music collection"; "Here I typed 'n'. I conveniently get an artist view of 'n'. (below images of all artists 4, 5)" (Source: Heesung Koo)

• 

  Virtual folder - all artists 4

• 

  Virtual folder - all artists 5: "Here I typed Nina for Nina Hagen. I conveniently get an artist view of Nina." (Source: Heesung Koo)

Microsoft also intended that Longhorn would support sets. A static set was intended to be a list of items stored within WinFS and that did not share common properties. For example, a music playlist is a list of pointers to actual music files stored on some physical media. The files could be stored in different locations, but the lists could be used across different computers, and users can control the hierarchy of the lists. Static sets would have been best used to organize items that hadno properties in common. For example, a user might maintain a list of documents he's working on. The documents have no properties in common.

In contrast, dynamic sets would have been the results of queries based on the properties of items (for example, image files or all files created in February) in WinFS that were run against certain locations (called the scope). The sets could be used across different computers. For example, scope could include the computer used by the user, a shared server, or a company's network domain. In WinFS, dynamic sets would have been the preferred way that users can organize their files—not by location, but by the properties most relevant to them.

Dynamic sets were intended to work as follows:

With "Longhorn," users perform familiar file management tasks such as finding, organizing, and performing actions on data. However, dynamic sets replace the need for physical folders and become the primary way users access their data.

In "Longhorn," the list view control shows the items that have been dynamically selected based on the user's criteria. Each item in a list view is not necessarily implemented as a file in the file system. A list view can contain contacts, e-mail messages, and other non-file objects. Depending on the selected view property, users can see previews and details about the files in the list. However, in icon views, for example, this amount of detail may be minimal.

Dynamic sets can be grouped visually into "stacks" according to a specific property such as author. If a user chooses to stack the items by author, a stack for each author is displayed, and items with no author value appear in a flat list. Dragging one of these items into the stack for a particular author assigns that author as a property of the item, which allows for easy property editing.

With filter menus, storage favorites, and the wordwheel, users can change the dynamic set they are viewing until they have narrowed the set to exactly the items they want to view.

Users can select a particular property value from a filter menu to narrow a dynamic set to items with that value. For example, a user could select Raquel Mello from the "Author" filter menu, and the dynamic list would narrow to only items created by Raquel Mello.

A user could further narrow the dynamic set to only documents created in a particular month.

The breadcrumb bar in the storage page helps users keep track of where they are and what filters they have used.

Users can save links to their favorite dynamic sets, called "storage favorites," which are readily available in the UI.

Clicking a storage favorite runs a query of all items within a specific scope, and returns only the items with specific properties. For example, a user might create a set of important documents in the month of October, and he might want to see this set of documents frequently. While viewing the set of documents, he could save a storage favorite and name it "Important Documents from October." This storage favorite would be immediately available to him in the future.

Users can also filter the items in a dynamic set using the wordwheel.

As a user types in the wordwheel, the dynamic set narrows to items with properties that match or contain the string that the user typed.

• 

  "Figure 1 Schematic of storage model" (Source: Microsoft User Experience Group)

• 

  "Figure 2 Sample storage page for Documents" (Source: Microsoft User Experience Group)

• 

  "Figure 3 A dynamic set" (Source: Microsoft User Experience Group)

• 

  "Figure 4 A dynamic set with stacks" (Source: Microsoft User Experience Group)

• 

  "Figure 5 Filter menus" (Source: Microsoft User Experience Group)

• 

  "Figure 6 Author filter menu" (Source: Microsoft User Experience Group)

• 

  "Figure 7 Date filter menu" (Source: Microsoft User Experience Group)

• 

  "Figure 8 Breadcrumb bar" (Source: Microsoft User Experience Group)

• 

  "Figure 9 Sample storage favorites" (Source: Microsoft User Experience Group)

• 

  "Figure 10 The wordwheel is the text box above the list of favorites" (Source: Microsoft User Experience Group)

Longhorn would have allowed a user to add a property to an item through the preview pane, by dragging an item into a stack, or through the property page.

• 

  "Figure 11 Editable property fields" (Source: Microsoft User Experience Group)

• 

  "Figure 14 Property page" (Source: Microsoft User Experience Group)

Users could form search queries using words that mean something to them, such as, "What is a firewall?" or "Pictures from Maui." Users could perform more complex searches, such as "Look for information about Jim from 1987 to 1999," because non-file-based data like contacts and e-mail messages were cross-referenced with files.

• 

  "Figure 12 Search page" (Source: Microsoft User Experience Group)

• 

  "Figure 13 Search results page" (Source: Microsoft User Experience Group)

The preview pane would have showed commands associated to a WinFS type or file type.

• 

  "Figure 15 Sample preview pane" (Source: Microsoft User Experience Group)

• 

  "Figure 16 Sample document commands" (Source: Microsoft User Experience Group)

A rules engine would have made possible for users to create simple or complex rules to define how their system handles particular messages.

• 

  "Figure 17 Creating a notification rule" (Source: Microsoft User Experience Group)

• 

  "Figure 18 Do Not Disturb settings" (Source: Microsoft User Experience Group)

• 

  "Figure 19 Notifications history" (Source: Microsoft User Experience Group)

Microsoft also intended that applications could easily reuse any contacts already present in the WinFS Contact store. Contacts would appear on the Contacts explorer, and for frequently communicated contacts, on the Sidebar. Contacts would also have their own property pages, with the following parts:

• The preview pane displays the commands with which users can perform various contact or communication tasks. You can add commands from your application to the command list in the preview pane.
• The explorer pane gives users a view of the data from a given application and provides links to related files, the communication history, and other applications that create property page extensions. If an application has synchronized data about a given contact into the contacts explorer, that data is listed in the explorer pane.
• The content area contains a composite view of contact data from multiple sources. Windows provides UI to alert users to and manage any conflicts between applications.
• Property page extensions allow your application to embed a preview of your content inside the property page. If you want your application to add its own propriety contact data, you can extend the contact schema and create a property page UI extension to render application-specific data inside the property page.

Messages in the WinFS Store would be displayed in Communication History. Users may access the history of multi-application communication messages related to a given contact.

• 

  "Figure 1 The contacts explorer" (Source: Microsoft User Experience Group)

• 

  "Figure 2 Contact text box" (Source: Microsoft User Experience Group)

• 

  "Figure 3 Contact picker dialog box" (Source: Microsoft User Experience Group)

• 

  "Figure 4 Contact controls embedded in a sample application" (Source: Microsoft User Experience Group)

• 

  "Figure 5 The contact property page" (Source: Microsoft User Experience Group)

• 

  "Figure 6 The contact sidebar tile" (Source: Microsoft User Experience Group)

• 

  "Figure 7 The communication history window" (Source: Microsoft User Experience Group)

Implementation

A Milestone 4 build that leaked in or around March 2003 contained "Windows Future Storage Services" and "Windows Storage Full-Text Index". A user opening Search would simply be asked: "What are you looking for?", with a list of example searches provided for reference such as "Pictures from John" or "What is a firewall?". The build came with libraries, stacks, and filter features.

• 

  My Contacts in build 3683

• 

  Picture & Video Library in build 4005 was powered by WinFS. Note the wordwheel, the text box below the words "Filter by:"

• 

  Music library in build 4008

• 

  Stacks in Music Library in build 4008. Note the quick links at the left and compare with the Longhorn Days demo.

• 

  Picture & Video Library in build 4015.

• 

  Picture & Video Library in build 4015 after pictures populated by WinFS

• 

  Game Library in build 4015

• 

  Music Library in build 4029

• 

  Game Library in build 4030

• 

  Picture & Video Library in build 4030

• 

  Game Library in build 4033

• 

  Game Library in build 4033, with context menu

• 

  Picture & Video Library in build 4033

• 

  Music Library in build 4033

• 

  Game Library in build 4039

• 

  Search in build 4039

• 

  Libraries in build 4039

• 

  Game Library in build 4040 in Carousel view

• 

  Game Library and Photos & Videos in build 4042.main

Build 4051, which was distributed during PDC 2003, contained WinFS. Part of the "Self-Guided Longhorn Tour" that Microsoft released for this build described how WinFS (erroneously called WinFX) worked, which operationalized the UI features Microsoft intended for Longhorn:

1. From the Start menu, click Computer. This is a place from which you can get to all of your storage locations, such as the WinFX^TM default store.

2. Open the "DefaultStore," which is a view of WinFX through the Win32® APIs. Drag in copies of a variety of photos, music, and documents. (Use only copies of your data to prevent data loss.)

• Because WinFX is still in development:
  • It might take several minutes to copy the files and promote the metadata.
  • You cannot rename items that you've added to WinFX.
  • Saving files might be an issue because of application compatibility.
    
    

3. Open the Start menu and click Music. The music you added to the WinFX default store appears in the music collection.

4. Click the column heading Artist and select Stack by artist. The items appear in stacks according to their artist metadata. Double-click a stack to see music by a specific artist.

5. Click the column heading Track Number. Pick one, and the list narrows to only items with that track number.

6. Notice how your navigations are tracked in the Address bar. You can click on any item in the address bar to navigate, or click the drop-down control to the right of each item to navigate to one of the item's peers.

7. Find the text box labeled Filter by on the left. This is called the wordwheel, and it allows you to quickly filter the items you see based on what you type. Type an artist's name and notice that, as you type, the items are filtered and only items related to what you typed remain.

8. In the Address bar, click the drop-down control next to Music and select Photos and Videos to navigate to the photos and videos collection. The photos you previously added to WinFX appear.

9. Use the Address bar to navigate to "Documents". Select a document, and a thumbnail of the document appears in the preview pane, along with relevant tasks and metadata for the item.

10. From the preview pane, click Add a category. Type categories related to the document (separated by semicolons) and press Enter. Because it's in development, this might take a few minutes.

11. Click the column heading Category and then select one of the categories that you added. The items will filter to only the item for which you attached this category.

12. Sign on to Windows Messenger. After a few minutes, your buddies will be added to WinFX so that they will appear in your collection of contacts, and in the Contact Picker.

13. Use the Address bar to navigate to Contacts. All your WinFX contacts, including your Messenger buddies, appear. Double-click a contact to view and edit details for the contact.

14. To add contacts from Active Directory to your contact collection, open the File menu and click Add contacts. This launches the Contact Picker. To find the contact you want to add, type a name in the Search For text box, and then click Search. Select the contact and click Add.

WinFS in build 4051 suffered from huge performance issues.

• 

  Computer in build 4051. Note the icons under Storage and labeled Type: Storage.

• 

  Contacts in build 4051 was powered by WinFS.

• 

  Rules and Alerts, one of the UI features intended for WinFS, in build 4051

• 

  Rules and Alerts, setting up a rule

Build 4074, the WinHEC 2004 preview build, also included WinFS. It slowed down the system because it catalogued every single file, including system data, so that WinFS would be able to search them.

• 

  Outlook Express in build 4074. Note the WinFS.exe CPU and memory usage on the Task Manager.

• 

  Communication History in build 4074. Note the WinFS.exe CPU and memory usage on the Task Manager.

• 

  Music in build 4093

Data storage

From this section, the discussion is on how WinFS worked under Beta 1, the latest release of WinFS to the public, not under any Longhorn pre-reset build. All documentation therefore should refer to the WinFS Beta 1 time frame, PDC 2005 included.

Architecture

What is WinFS?

WinFS used a subset of SQL Server 2005 technology, modified for file storage. It consisted of the following components:

• "WinFS" Core - the core services expected from a file system, and included the relational storage engine;
• Data model - the technical innovations of WinFS, including basic item structure, relationships, and the ability to extend both items and relationships;
• Schemas - built-in schemas allowed WinFS to understand a user's data in richer ways than traditional file systems and provide a more structured way to handle metadata. Included were first, schemas for everyday information such as documents, e-mail, appointments, tasks, media, audio and video, and second, system schemas that define configuration, programs, and other system-related data;
• Services - included syncronization and rules (discussed below);
• APIs - which developers might have used to program the different building blocks of the "WinFS" system, including data operations, rules, synchronization, and the data model

WinFS required three services, which ran in three processes:

1. WinFS.exe, the WinFS data store, that ran a Yukon (SQL Server 2005)-derived database engine
2. WinFSSearch.exe, which ran the WinFS search engine
3. WinFPM.exe, which ran the WinFS File Promotion Manager. File promotion referred to the process of extracting data from a file and populating the data to an item. The reverse process of persisting itemized data to streams of the underlying file was called demotion. WinFS served all kinds of data: structured, semi-structured, or unstructured, through a new file system driver (RsFxDrv.sys). This driver allowed a user to access a WinFS item both as a file stream (as a file) through the Win32 API or as an item via the WinFS API. The process of file promotion and demotion kept the file-based and item-based representations of a WinFS item synced. This made WinFS backwards-compatible with NTFS.

WinFS depended on the .NET Framework, and used the same APIs as other data types thanks to ADO.NET. The WinFS API was included in the System.Storage namespace.

Data store

WinFS data hierarchy

WinFS kept data in Stores, which in turn are kept in a relational backend. A WinFS store was a common repository where any application can store items and associations. Items could either be backed by files (for example, images and documents) or not (for example, contacts and e-mail). Associations established relationships between different types of data (items), which then allowed users to organize information based on how they used them. Bagnaru explained how items and associations were stored:

Q: Mikael writes, “How and where are all the relations, keywords, etc. saved?”

A: The item properties are stored in the WinFS store as part of the item itself. WinFS has a concept of “links” which are an entity type with a source item and a target item that relate items. There is also the concept of “common value” relationships which are based on item properties. Navigation of common value relationships is a value based join.

All data in the store were accessible as CLR objects, allowing a developer to leverage all their Visual Studio and .NET knowledge while building apps..

WinFS, by default, came with two stores, DefaultStore and Catalog. DefaultStore was the default location for user and application "WinFS" data. Catalog stored metadata about "WinFS" stores and shares, as well as other system data. The Catalog store was read-only and should not be used for user data. Users could create additional stores, but a store could not span multiple volumes and could only be hosted on NTFS volumes. The reason for these limitations was that every WinFS store had a root share with the same name as the store.

WinFS stores were made up of database files stored in a directory under the "System Volume Information" folder. Filestreams were saved as ordinary NTFS files with a GUID as file name. Tony Voellm of the WinFS Team explained, "The term NTFS Streams was used but the File Data is actually stored in NTFS files." Meanwhile, metadata is stored in a SQL Server database.

Data model

WinFS Types

The WinFS data model is the architecture of the type system. WinFS introduced a type definition language, an XML vocabulary but not the XML Type Definition Language, to describe types.

WinFS provided a set of schemas that defined types and attributes for implementation by the WinFS API. A developer could also create custom schemas.

A schema was a collection of type declarations, such as the declaration of a postal address, a full name, or a person. Each schema defined a set of types that mapped to some information. All these types derived from a set of base types defined by "WinFS". The base types grouped related types together, creating type families.

The root types in the WinFS system were Item, Link, InlineType, and ItemFragment. The other WinFS types derived from the root types. Descendants of one of these types were entity types. For example, Person was derived from the Contact type, which derived from the Item type. AttachmentLink was derived from the Link type. Therefore, Person, Contact, and AttachmentLink were all entity types. An EntityType had its own unique identity, and so could exist on their own. In contrast, an InlineType did not have a unique identity and could exist on its own. FullName of a Person is an InlineType, a Person is an EntityType.

ItemFragments were also entity types, which enabled the declaration of large collections in the item types, such as multivalued properties. Each collection member had a key, which enabled applications to access individual members of the collection independently of the owning item. An example would be a message, which might have a large number of participants. The ItemFragment would be participants under the Item message.

Meanwhile, an EntityExtension allowed a user to extend built-in items by adding further properties as necessary. WinFS schemas defined several extension types. For example, the System.Storage schema namespace defined two ItemExtension types: Reminder (such as a birth date Reminder to a Person) and ItemKeywords (which allowed a user to assign keywords, such as "legal document" or "personal document" to a Document).

Links and associations

Links referred to data that defined links between two or more distinct existing item types; for example, an Item type, Message, might have attachments that were documents, another Item type. WinFS also provided the concept of Associations to relate items. Associations were not physically stored in the store. An example of an Association would be that declared by a user between the Document and Contact types when identifying the publisher of a document.

WinFS depended on applications and users to provide the information to create links. According to Bagnaru,

Q: Mikael also asks, “Do I have to set all relations manually? How does WinFS know which document is created by which user?”

A: The relationship “setting” will depend on how it is modeled. A link will need to be set by something (i.e. metadata handler, sync adapter, application, etc.) A common value relationship, however, is just based on a join. WinFS depends on the application and user to provide the information to create links. In many cases, an application already has this data; for example, Office applications already have an “Author” property. In other cases, the user already knows this information and can set it. It would be fairly easy to get my primary contacts’ information moved into WinFS and then relate pictures to the contacts. In the future there could be many processes that could be used to relate your data: natural language processing of documents, concept extraction, face recognition for images, etc.

Item

The Item was the fundamental concept in "WinFS". An item was the smallest data element defined by WinFS, and was the equivalent of an object in object-oriented programming. Each item had a unique identifier, its GUID.

Items grouped together in DefaultStore

WinFS defined common item types, which resided in the following WinFS namespaces:

• System.Storage.Audio
• System.Storage.Calendar
• System.Storage.Contacts
• System.Storage.Documents
• System.Storage.Image
• System.Storage.Media
• System.Storage.Messages
• System.Storage.Video

An item that would fall under System.Storage.Documents would lead to, as Malik explained:

For instance, if you author a Microsoft Word document, it is automatically promoted to an instance of System.Storage.Documents.Document, which then automatically populates its various relevant properties such as title, subtitle, author, etc.

If a file did not fall under any of the common item types, WinFS stored them as a GenericFile item. This happened when no file metadata handler was registered for a file name extension. A file metadata handler promoted and demoted properties between structured file formats and "WinFS" items. (See #Architecture for discussion of file promotion and demotion.)

A definition for item types then reduced the amount of code to bridge between types. Grimaldi showed the benefit of this:

For example, if your application and my application both use the WinFS person type, no transformation is necessary. Even if I extend the WinFS person type, we may be able to share the data without transforming it. For example, if I write an extension to the person type that adds a property for their medical insurance member ID, and your application is a greeting card application, I can pass a person between our applications with just a cast to person.

Another benefit was common behaviors. Only one code would be necessary to save an item, regardless of type.

Defining common item types also allowed WinFS to look inside files in a similar manner to desktop search engines. Bagnaru pointed out:

Q: Lars asks, “Does WinFS look inside files (word, excel, pdf, etc.) like desktop search engines do?”

A: Yes. When a file is stored in WinFS, its metadata will be extracted by type specific property handlers. Part of our Beta 2 work is to integrate this with the Windows Desktop Search handlers. The end result is that a Windows Desktop Search query will return WinFS and non-WinFS files.

Rules

WinFS Rules provided an object-oriented API useful in definition of data sets, logical properties of data, and associations between data items.. The WinFS SDK Beta 1 Documentation defined a rule as conditional logic used to define queries, filters, computed properties, or computed associations that can be composed, persisted, and reused.

WinFS Beta 1 supported the following rules:

• FilterRule - A rule that contained filtering logic that could be applied to item(s) of the given type. FilterRules obtained derived item sets based on the conditions of the rule;
• PropertyRule - A rule used to calculate custom properties from existing properties of data objects. For example, a PropertyRule could define a new property of the Contact class, named locale, based on the zip code of the contact's primary address. Property rules allowed a developer to define new properties on item types without modifying or extending the schema for that type. To illustrate, an email application could expose capabilities for creating a “Color” property for messages and allow each user to define the value of “Color” using their own logic;
• AssociationRule: A rule to calculate relationships between data objects. For example, a Document could be associated with a Contact defined as its author or copyright owner;
• ActionRule: A rule to identify actions available on various data types. Rules of this type identified actions to be performed on data items. For example, the Photo class could define resize or rotate actions;
• Query: A rule that defined a derived set. It was used to implement execution of a query as distinguished from the definition of a rule.

Kati Dimitrova, a developer on the WinFS API team, explained how a query worked as a rule through a scenario involving Mary, a recruiter trying to manage and track candidates throughout her day across multiple applications:

First and foremost, WinFS rules are persistable units of logic. In this scenario, Mary creates the query “candidates that are interviewing today” through her WinFS-enabled recruiting application and persists it. But how is the query persisted? As a WinFS item. In fact, all WinFS rules, including queries, are items. This has many advantages. To start with, rules can be created, manipulated, stored and secured in the same manner as all other items.

Further, WinFS rules are about discovery and sharing of persisted logic among applications. Because rules are WinFS items, they can also be queried for, discovered and shared among applications just like all other WinFS items. In this scenario, Mary can easily find and use the query “candidates that are interviewing today” from her email application even though she created it through her recruiting application.

Finally, WinFS rules are about re-usability and composition. Not only can Mary find the query she created in her recruiting application, she can re-use it in her email application to create a new query “mail related to candidates that are interviewing today” that refers to the original query. Note that composition of rules is true composition; if Mary changes the logic that defines the query “candidates that are interviewing today,” those changes will affect the results of the query “mail related to candidates that are interviewing today.”

Access control

Bagnaru explained how WinFS secured data:

Q: Chris asks, “While it's good to have a common silo that all apps can easily use, it's seems a bit risky. Wouldn't some rogue app/spyware have an easier time getting to financial Quicken/Money data?”

A: Even though all your data is in a single silo, all the data isn’t equally accessible. WinFS provides two data protection mechanisms. First, there is share-level security that controls access to your WinFS share. Second, there is item level security that supports NT compatible security descriptors. In order to access an item, the caller must be provisioned at the both share and the item. To manage this, WinFS will provide a rich API for security administration. Also in Vista there is the concept of “integrity level” for an application (Mandatory Integrity Control). Data can be configured to prevent lower integrity apps from accessing it. Simon (our security PM) will post more detailed description about this soon.

Data retrieval

Searches and queries

WinFS used OPath, a string-based query language used for filtering, sorting, and projecting information stored in the WinFS store. OPath was used in multiple places on StorageSearcher to specify queries, and in other places in the API, such as when creating StorageViews.

OPath allowed a user to perform the following types of queries:

• Extent - Identified the source of the data queried in an OPath query expression;
• Filter - Created a collection of objects that contained a filtered set of objects from the input collection;
• FilterByType - Created a collection of objects that contained only the objects from the input collection that were of the specified type, or a sub-type of that type;
• Group - Created a collection of objects that contained groupings of the data in a collection of objects;
• Join - Created a collection of objects that contained parings of data from two or more input collections of objects.
• Project - Created a collection of StorageRecord objects that contained only the selected properties of the objects in the input collection;
• Sort - Created a collection of objects with the data sorted by one or more property values of the input collection;
• Top - Created a collection of objects that contained the first number of objects in the collection specified;
• TreatAsType - Created a collection of objects that contained objects that were converted to a base type
• Union and UnionAll - Created a collection of objects that contained the data from two or more collections of objects;
• Yield - Created a collection of values computed by applying the OPath expression specified to each object in the input collection

Notifications

A notification was simply defined as "TBD" in the WinFS SDK Beta 1 Documentation.

Data sharing and synchronization

WinFS Beta 1 included a Windows Shell namespace extension, which showed WinFS stores as top level objects in "My Computer". The stores also appeared as a share in "My Network Places". Files could be copied into and out of the stores. Applications could be directly used to save there through the share \\COMPUTERNAME\DefaultStore in "My Network Places". Even folders such as My Documents could be redirected to a WinFS store.

• 

  WinFS Stores top-level object in My Computer

• 

  WinFS Stores, note "DefaultStore"

• 

  DefaultStore in a fresh install of WinFS Beta 1

• 

  DefaultStore top-level object in My Network Places

• 

  DefaultStore views from My Computer (left) and from My Network Places (right)

WinFS Sync allowed synchronization of data between WinFS stores and non-WinFS, or external, data stores (for example, synchronizing contacts between a "WinFS" store and a PDA or synchronizing e-mail between "WinFS" and an e-mail server) and synchronization of data between "WinFS" stores (for instance, making "WinFS" data available on all computers in a home network or sharing a set of documents, peer-to-peer, at work).

WinFS Sync provided change enumeration services that used change tracking to determine all entities that had changed since a previous synchronization. It also provided change application services that made changes to the data in WinFS.

WinFS allowed for both one-way importers, moving data from an application to WinFS, and two-way importers, which allowed for both. WinFS could resolve conflicts among data. Neil Padgett, program manager working on the WinFS Synchronization APIs, explained:

Let’s assume that we did some initial sync to ensure we had our contact in both stores. (We can talk about what this exactly means later, but for now we can think of it to be like running our importer.)

And let’s imagine that, after we sync we make some updates:

In the application store, we’ll update:
-Home address
-Telephone

And, in WinFS we’ll update:
-Home address
-Cell

So, now imagine we want to bring our two stores back into sync. So, how do we do that? We know we were in sync before, so we need to figure out what changed on each store, and then apply those changes to the other store. We can call these processes change enumeration and change application, respectively, and we want to do them in both directions. (In fact, WinFS does the hard work of figuring out what changed for us and of making sure that remote changes brought to WinFS are not echoed back to us later.)

Considering our example, we have some changes that are straightforward – the telephone numbers were each changed on the one store, but not on the other. We call these non-conflicting changes. For these non-conflicting changes, we can simply apply them each to the other store. The difficulty comes with the home address – we made changes on both stores – so-called conflicting changes. We’ve detected a conflict and we’ll need to resolve it, either by prompting the user or, more likely, according to some policy (for example, keeping the latest change.) Then, we can bring our two stores back into sync.

— Neil Padgett, Getting Data Into WinFS with WinFS Synchronization,

Applications

WinFS Beta 1 shipped with unsupported applications: File System Sync Adapter, Microsoft Rave, StoreSpy, Type Browser, and OPather.

File System Sync Adapter

The File System Sync Adapter synchronized file-backed items and directories between a WinFS replica and a file system directory.

Microsoft Rave

Microsoft Rave attempting to join synchronized folders "A" and "B". Note the "File System: WinFS" that appears in the Explorer window of DefaultStore mapped as a network drive

Microsoft Rave allowed synchronization of two or more WinFS stores. A user could synchronize between WinFS folders in the same machine or in multiple computers, and also with other users in a Windows domain.

Microsoft Rave used peer-to-peer technology to communicate and transfer data.

StoreSpy

StoreSpy included in WinFS Beta 1

StoreSpy allowed a user to search for files in stores by executing queries. It supported type-specific grid views, reminiscent of Outlook 2003, for Documents, Messages, Events, Contacts, and Photos, with a typed preview pane for messages, meetings, contacts, and photos, and a debug preview pane for other types.

StoreSpy supported property, query, full-text, and folder based filtering. It also allowed saving and retrieving queries. It also included a one-way Outlook sync adapter to bring items from Outlook 2003. However, it could only read from a Global Access List.

StoreSpy did not allow editing of items or their properties, though it was slated for inclusion in a future release. WinFS was cancelled before it could materialize.

Shishir Mehrotra demoed StoreSpy on a video uploaded on the Microsoft Developer Network's Channel 9.

An earlier version, v0.8, was released for the WinHEC 2004 Longhorn build, 4074. It was an Explorer-like tool to browse WinFS, and allowed a user to inspect Item, Relationship (all 3 types), Extension, Nested Element, and MultiSet properties.. However, the download link has not been archived.

WinFS Type Browser

WinFS Type Browser could be used to visualize WinFS schema types (not WinFS data which StoreSpy showed), their type hierarchy, methods, properties, other types it was related to, among others.

Shishir Mehrotra also demoed the WinFS Type Browser on a Channel 9 video.

OPather

WinFS Beta 1 also included OPather. Luiz Miranda of the WinFS Team explained:

When I created the tool I wanted a GUI interface where I could quickly set my target object and start writing my query and know right away of any mistakes in my query string. To do this with OPather is a snap. Open the tool, select the target object type you want to start querying against, then start writing your OPath query. There is even an Intellisense-like feature to help guide you through simple queries. Combined with other features that grew from feedback such as binding results to a DataGrid, constructing StorageViews, and ability to Dump all Code for written query, this tool can certainly help beginners and advanced users who intend to write an App that queries against the WinFS Store.

Project "Orange"

Microsoft launched a team, codenamed Project Orange to build an application built on the Windows Presentation Foundation ("Avalon"), that would demonstrate some of the ideas showed in the IWish video from PDC 2005. (discussed above at Development). Shishir Mehrotra of the WinFS Team shared some links to job postings and quoted some job descriptions covering positions at Project Orange.

However, since WinFS development went dark, the status of Project "Orange" is unknown.

External links

• WinFS: Bill Gates' biggest disappointment
• WinFS at Wikipedia. This BetaArchive Wiki article was started as a fork from its Wikipedia counterpart with revision date of 5 January 2021.

Longhorn-era WinFS

Microsoft

• WinFS: The Windows File System (Microsoft Access 2002 Technical Articles)
• User Experience (Aero): Contacts (Microsoft Access 2002 Technical Articles)
• The Pillars of Longhorn - from the Microsoft Developer Network (MSDN)
• The Pillars of Longhorn: Living La Vida Longhorn (Longhorn Foghorn) (MSDN)
• Chapter 4: Storage (MSDN)
• WinFS Lets Users Search and Manage Files Based on Content | Microsoft Docs
• Data Access and Storage Developer Center: Building WinFS Solutions - WinFS Longhorn era on MSDN
  • Wayback Machine list of all Longhorn era WinFS documentation archived from http://longhorn.msdn.microsoft.com/lhsdk/winfs/
• Understanding Windows "Longhorn" - Official Microsoft links by Ian W from Neowin.net, includes Wayback Machine links for Microsoft resources on WinFS

Non-Microsoft

• MS poised to switch Windows file systems with Blackcomb
• XP successor Longhorn goes SQL, P2P – Microsoft leaks
• Windows on a database – sliced and diced by BeOS vets
• New Windows could solve age-old format puzzle--at a price, with timeline of Cairo's OFS
• Get a Grip on Longhorn (Figure 5. List WinFS Contacts in a WinForm)
• Microsoft geeft videopresentatie over WinFS - Microsoft provides video presentation about WinFS (Dutch) - Download page from the Wayback Machine of the video: MSDN TV: WinFS Overview - Significant New Storage Innovation
• Code name: Longhorn by John Clyman, PC Mag article
• Guide: WinFS and tasks in build 4042 | Microsoft Longhorn
• Windows LGH 4051+WinFS - YouTube
• Windows Longhorn 4074+WinFS - YouTube
• An Opinion, blog by Mike Deem, Program Manager working on WinFS circa 2003, contains posts on WinFS

Reactions to WinFS delays

• New file system has long road to Windows
• More news on WinFS..., in response to "New file system has long road to Windows"
• Shipping WinFS, in response to the links above ("New file system has long road to Windows" and "More news on WinFS..")
• Proudly Serving My Corporate Masters: Scheduling and Black Hole Projects, connected to links above ("New file system has long road to Windows", "More news on WinFS..", and "Shipping WinFS")

WinFS Beta 1 era

Microsoft

• WinFS Team Blog
• WinFS on MSDN - WinFS Beta 1 era
• Looks like WinFS is back on track
• WinFS Beta 1 Released (video soon to come to Channel 9)
• The WinFS Files: Divide et Impera

Non-Microsoft

• WinFS Beta 1 and Microsoft’s Search Strategy
• First Impressions: WinFS Beta 1
• WinFS Beta 1 - The testing continues...
• WinFS Beta 1: Useless?
• WinFS Basics: Working with a Data Store
• JD’s Weblog » WinFS Resources
• Trends in Database Development: XML, .NET, WinFS
• WinFS Overview - NTFS.com (first of a series, see Contents section)

Hal Berenson: WinFS, Integrated/Unified Storage, and Microsoft

• Part 1
• Part 2
• Part 3
• Part 4

General links

• Channel 9 Videos
• WinFS Newsgroup
• WinFS area on NetFXGuide.com
• WinFS Forever- 2010 | SwimmingPool.Net (features intended for WinFS that made it to Windows Vista and 7)
• What WinFS is all About | The NeoSmart Files

BetaArchive

• WinFS Beta Refresh on Vista/XP - Possible? Useful? - BetaArchive
• Question: Longhorn 4079 - WinFS Storage fold. in my computer - BetaArchive - thread is about 4074; the post by Smartie77 explains how WinFS works in the build
• Impact of release of WinFS on computing? - BetaArchive
• Static List? - BetaArchive - UI feature for Longhorn-era WinFS
• Original Patent for Longhorn - BetaArchive
• 4015 playlist maker - BetaArchive
• Breadcrumbs in Windows Vista Developer Story - BetaArchive - Breadcrumbs were intended as a UI feature for Longhorn-era WinFS, see Development
• Communication History in Longhorn - BetaArchive
• WinFS: A pillar of Windows Longhorn and Bill Gates' biggest disappointment - BetaArchive

Other forums

• The WinFS File System For Windows Longhorn: Faster & Smarter – OSnews
• Longhorn build 4074 and default store
• Using WinFS in Codename Longhorn 4074 : General Windows Support
• WinFS Beta 1 Released Early - Slashdot
• WinFS Beta 1: How useful? - AeroXperience
• WinFS ready applications? - AeroXperience - 2,000 Members =)
• AeroXperience -> WinFS (Older version)
• News: Microsoft kills WinFS project
• So what happened to WinFS? : General Windows Support
• I need to un-install .NET2.0b : General Windows Support - Uninstalling WinFS also uninstalls the .NET Framework 2.0 beta
• WinFS - what it could have been, and why it failed | MetaFilter
• filesystems - What happened to WinFS? - Super User
• Who Needs Git When You Got ZFS? (2013) | Hacker News, with discussion on WinFS