What’s in an SDK?
Couchbase Server offers multiple services for storing, retrieving, querying and analyzing your data. Each service exposes a public API either as REST API or via lower level, extended, Memcached binary protocol. With the proper Authentication and open ports, any application can use these APIs, but they come at a cost, complexity and lack of type-safety for the most part. Let’s face it, most application developers are not concerned with parsing a REST response or un-packing Memcached packets; they want to quickly and efficiently build their solutions to solve their unique business needs. They want a library that handles resource management, sharding, managing cluster topology changes, etc. which is the role that a Couchbase SDK serves.
Why are we changing?
SDK 1.0 aligned with Couchbase Server 1.x and supported mainly read/writing of Key/Value pairs in a binary and JSON format using the Memcached protocol and Views. The SDKs were developed independently of each other and lacked a consistent API interface. In some cases, Memcached packing differences made the data un-shareable across SDKs.
SDK 2.0 supported new features of Couchbase Server as it evolved: N1QL, the SQL-like language for querying JSON documents stored in Couchbase Server, improved authentication using Role Based Access Control (RBAC), improved JSON support, Analytics, Full Text Search (FTS) and other tools for interacting with your data. In 2.X we aligned interfaces and made them consistent, for the most part, across all SDKs using an RFC to ensure adherence to the defined interfaces and included a Common Flags component for ensuring data was writable and readable from any SDK. On the downside, the IBucket interface become bloated and difficult to use as multiple APIs were crammed into it.
SDK 3.0 aligns with new features to Couchbase Server that extend from the traditional Bucket interface: support for Scopes and Collection that will be available as a Developer Preview in Couchbase Server 6.5 and as GA in Couchbase Server 7.0. Additionally, it consolidates and refines the formally bloated, overload based interfaces into a smaller, more concise interface while retaining and improving cross-SDK conformance.
How do we Version?
All SDKs subscribe to the Semantic Versioning 2.0.0 Specification (SemVer) where for any given version number MAJOR.MINOR.PATCH is incremented when:
- MAJOR version when you make incompatible API changes,
- MINOR version when you add functionality in a backwards-compatible manner, and
- PATCH version when you make backwards-compatible bug fixes.
Additionally, for pre-releases and build metadata we add extensions to the SemVer Specification as allowed by it. For example, our early drops of SDK 3.0 use the “alpha” designation plus an increment for the alpha version. For example, our initial SDK 3.0 early previews used the following SemVer compliant version: “3.0.0-alpha1”. The next pre-release after the alpha releases, will be “3.0.0-beta1” and finally a fully supported GA release will be “3.0.0”. In general, expect breaking changes between “alpha” versions, which should not happen (but in some cases may be necessary) once “beta” is released.
Note that this is a slight discrepancy between server version in that Couchbase Server will offer pre-releases as “developer previews” or “dp” and then “beta” and finally GA.
Scopes and Collections in Couchbase Server 6.5+
In Couchbase Servers versions before 6.5, Buckets were used as a logical, user-named entity that groups items; allowing them to be accessed, indexed, replicated, and access-controlled.
This was really your only means of achieving multi-tenancy using Couchbase Server and came with some limitations: Buckets themselves are fairly resource intensive and a cluster can only efficiently manage a finite number of buckets. For modern microservice architectures or really any architecture where multi-tenancy was needed, this made for some challenges when serving a large amount of tenants. This is solved in Couchbase Server 6.5+ by Scopes and Collections.
A Scope in Couchbase 6.5+ represents a unit of multi tenancy and allows Collection names to be reused; any given Scope can contain exactly one unique Collection name. A Scope is simply a group of Collections. Every Bucket contains a default Scope with the name “_default” and a Scope Identifier of 0 (zero) which contains a default Collection named “_default” and its accompanying identifier of 0 (zero). Of course, for every given Scope multiple Collections can be created.
This has the effect of moving Key/Value operations which were in the Bucket context for SDK1.0 and SDK2.0 into the Collection context for SDK3.0. Whilst, cross Bucket operations such as FTS, Analytics and N1QL are now done at the Cluster level.
.NET SDK 3.0 Alpha Series
The first package dump of the Couchbase .NET SDK 3.0 was Couchbase Alpha 2 (Alpha 1 contained a bug that made it necessary to release and bump up the version). Alpha 3 was released on April 26th – we plan on releasing several more! The focus on a pre-release is to provide the bits to program Key/Value operations and N1QL queries against Couchbase Server. It contains the familiar Cluster and Bucket logical structures as well as the new Scope and Collections structures. Note that while Scopes and Collections are part of Couchbase 6.5 that will not be released until later this year or next year, SDK 3.0 also supports Couchbase 5.0 and higher by using the default Scope and Collection explained earlier.
Cluster and Buckets in SDK 3.0
Just like SDK 2.0, Cluster and Bucket objects are back in SDK 3.0 each representing their server equivalences. Connecting to a Cluster and opening a Bucket are done in a similar manner to SDK2.0:
|
1 2 3 4 5 6 7 8 |
//Configure and open a cluster object var cluster = new Cluster(new Configuration(). WithServers("couchbase://localhost"). WithBucket("default"). WithCredentials("Administrator", "password")); //open a bucket named “default” var bucket = await Cluster.Bucket("default"); |
Similar to SDK2.0, the Cluster and Bucket objects are long lived objects whose life generally spans from application starts-up until the application shuts-down.
Using the default Scope and Collections in Couchbase Server 6.0
All Key/Value operations now exist on the Collections level, which will always be a member of exactly one scope. Here are a couple of examples of accessing a Collection and then reading and writing data into the Collection.
|
1 2 3 4 5 6 7 8 9 10 11 |
//open the default collection/scope var collection = await bucket.GetDefaultCollection(); //add a new document await collection.Insert(key, new {name = "mike"}); //retrieve the document using (var result = await collection.Get(key)) { var content = result.ContentAs<dynamic>(); Console.WriteLine(content.name); } |
Note that we are using Couchbase Server 6.0 and thus we don’t have Scope or Collection support (they will not be available until Server 6.5), however, we can use the “default” Scope and Collection as if we do have support for Scope and Collections. Note that this was added as an upgrade feature for migrating towards the newer server version. The “default” Scope and Collection will always be available; its similar to how a bucket works in pre-3.0 SDKs.
Note that after fetching a document, the GetResult must be disposed to free up resources otherwise the internal buffers will be “leaked”; this is a change from SDK 1.0 and 2.0.
Using the named Scope and Collections in Couchbase Server 6.5+
This is the future of Couchbase; the ability to slice a Bucket into multiple logical Scopes (Multi-tenancy) and Collections of distinct documents. This replaces the requirement in early versions of Couchbase server to add a special “type” field which designated a sort of grouping of documents by name which could then be queried independently within a Bucket. I think you’ll find it’s a much more intuitive programming model and will enable future advances of server features in the future.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
//Open a Scope and then a Collection var scope = await bucket.Scope(“ACME”); var collection = await scope.Collection(“tools”); //add a new documentawait collection.Insert(key, new Tool(Name = "Hammer", Type=”Ball peen”}); //retrieve the document using (var result = await collection.Get(key)) { var content = result.ContentAs<Tool>(); Console.WriteLine(content.Name); } |
Using the Cluster for N1QL Queries
N1QL, like Analytics and FTS, are considered “global” scope in that a query can reference multiple Buckets. Initially in SDK 2.0, N1QL queries were scoped to the Bucket and later added to Cluster, in SDK 3.0 you can currently only query from the Cluster object.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
//execute a query against the cluster var statement = "SELECT * FROM `default` WHERE type=$name;"; var result = await cluster.Query<dynamic>(statement, parameters =>{parameters.Add("name", "person");}); //enumerate through the rows returned foreach(var row in result){ Console.WriteLine(row.name); } //free up the internal buffers result.Dispose(); |
Overall, all of the services (Query, FTS, and Analytics) have a public API very similar to SDK 2.0, however, for consistency the required field (the SQL statement for N1QL) has been added as a parameter on the left-hand side and the optional parameters are now consolidated into an “options” block or structure on the right hand side. You will notice that this continues across the entire SDK and greatly improves the consistency of the API.
Getting the Couchbase .NET 3.0 SDK Alpha Releases
As with all Couchbase .NET SDK releases, we offer either direct downloads of the NuGet package and will also publish to NuGet.
Lastly, a special thanks to Brant Burnett of Centeredge Software for contributing massively to the release!
3.0 means your are allowed to have breaking changes?
Why not follow the convention about suffixing methods with Async when they are Async?
Hi Thomas –
Yes, there are lots of breaking changes – the public API is completly rewrite from SDK 2.X.
The rational for not using the “Async” suffix is because there is no sync methods – so no reason to differentiate between synchronous and asynchronous calls. Note that this may change in future releases as we get more feedback like yours.
Thanks for the feedback!
-Jeff