Support for PHP
Update 22 feb 2012Over the last couple of years I have been using NFileStorage for probably every project. I recently started to implement services in PHP (Wordpress) too and immediately felt the urge to use NFileStorage (PHPFileStorage ;) there too. For those who are interested in the PHP implementation please drop me an email. You can contact me via the email address which you can find on this page:
http://nexusstudios.nl
Summary
Update 5 june 2010As you might have heard Google has introduced Google Storage. When watching the video today (
http://www.youtube.com/watch?v=GLbRvcbkAwE) from the Google I/O 2010, I found out that the concepts / basics of NFileStorage are very similar to Google Storage. When I have time, it might be very appealing to upgrade NFileStorage to become compatible with Google Storage; thus, the end-user might have the option to choose to push data from a local filestorage to Google Storage, and to use NFileStorage to operate with cloud-storage.
DescriptionNFileStorage is a project (100% managed C# code) that enables software developers to easily handle large amounts of data (pictures or files in most cases) in your project without using a database and without having to store tons of individual files on the file system. It also ships with a CLI tool that enables command prompt access to all features.
Key reasons why to use it
- Persist any item from anywhere to your local file storage
- Zero configuration
- Serverless
- Easy access (use static methods and functions of a Facade to acccess all the features)
- Introspect the filestorage using a CLI and GUI
- Hook custom metadata to each item you store
- Save time when deploying and backing up releases
- Organizes your resources in a structured way
- Its free :)
Current statusThe FileStorage's primary features have been implemented and work stable. Take a look at the picture below where you can see a screenshot from one of the other projects I am working on (
prijs van een huis) that uses NFileStorage. This screenshot shows a web application that uses a FileStorage that is filled up with 150.000+ photos consuming over 700 MB of picture data. In the browser we can see a subset of the items that are streamed in realtime directly from the storage to the client's browser. The filestorage takes care of all the plumbing for us and nicely bundles all files in an organized way.
Functionality
- The data CRUD operations
- store any data in the storage identified by a global unique identifier (GUID)
- store files directly from the web (http)
- store files from your local file system
- store a stream
- store a list of bytes
- store a string (using specific or default Encoding)
- store objects (dependent upon the IDynamiteXml interface)
- extract data from the storage
- export data to your file system
- retrieve data as a string (using specific or default Encoding)
- export a single file to the file system
- retrieve list of bytes from a file in the file storage (for example to expose a file (picture, pdf...) to a web client)
- retrieve data as objects (dependent upon the IDynamiteXml interface)
- metadata storage
- list all items within the storage
- check if a specific item exists in the storage
- delete data from the storage (without purging)
- storage operations
GUI
Showcase
FAQ Frequently asked questions
Technical docs
FileStorage internals
Performance
Roadmap
Scheduled functionality for future versions
- a GUI to interact with the NFileStorage (scheduled for v1.3.1)
- caching concept for FileStorages that are static / read-only (scheduled for v1.3.1)
- improved file retrieval speed by optimizing some lines of code (in the index routines)
- allow caching of indexes to speed up data retrieval
- instead of reading each allocation block of the index, we could read up the entire index (assuming its not too big), and enumerate in mem, rather than on filesystem.
- make indexing logic pluggable
- optional a hierachy of files could be supported (oposed to the current approach where all files are stored without a folder structure).
- optional authorization concepts
- optional encrypted storage and decrypted retrieval
- optional on the fly compressing and decompressing of data (although not a big prio; as a) most file formats have some form of build-in compression, like jpeg, xdoc, and b) HD size, and network bandwidths are growing rapidly)
- additional features to manage a filestorage;
- feature to purge deleted items (remove older versions of an identical fileidentifier that was overriden) to reduce the data size
- 'rename' feature, allowing a dataidentifier to be renamed (easy, by invoking a clone and a delete)
- 'clone' feature to replicate a dataidentifier in the filestorage (low)
- 'cleanup' feature, allowing a filestorage to be cleaned up (to purge deleted items)
- 'allocate' feature, allowing the allocation of a certain amount of space for the data, which will be used to allow overriding of data that has expanded, without appending the data at the end of the data file
- feature to retrieve older versions of a specific fileidentifier, if present
- support for fixed-length file storages, where each file will be considered having a fixed length
- feature to allow overriding the actual data in the data store, which was already allocated by a fileidentifier, without appending to the end of the data block (limitation will be that the new data is not allowed to be larger than the initial one). Once this feature is in place we could also store the number of files in the data file itself (which could assist in making an auto incrementing number of unique GUIDs)
- storing meta data with statistics about the usage of the file handler (how often was it invoked, etc.).
- optimize by the fact that disks read chunks of 16K, 32K, etc at one time (even if you only want to read 1 byte).
- possibility to overwrite files in the storage
- make a FileStorageHandler bindable, to directly allow it to be hooked to a ListView
- expose events like BeforeItemAdd, AfterItemAdd etc.
- make plugins for nant (wouldn't it be great if you have a storage available in your nant scripts to hold variables, files, etc.)
- TransactionScope support in file operations over storage (requested)
Related / referenced projects
Alternatives