See the latest new features, improvements, and product updates
We continue to expand the list of destinations that you can setup directly in the Management app. Today, we have added two new destinations:
1️⃣ Elastic Search
2️⃣ Clerk.io
Read more about how to use these destinations in our documentation.
Only last week, we announced the availability of creating and managing the Algolia, Azure Service Bus, and Webhooks destinations via the Management app. Today, we are expanding the list of destinations to include:
1️⃣ Relewise
2️⃣ Typesense
3️⃣ Elastic App Search
Read more about how to use these destinations in our documentation.
We're thrilled to announce this update: Destinations can now be configured via the management app 🎉
What are Destinations? They're your gateway to effortless integration with third-party services like Algolia and for custom integrations with Azure Service Bus, and Webhooks. With Destinations, pushing data where it's needed has never been easier.
Previously a feature by request, Destinations are now available to all tenants via the Enterspeed Management App.
Curious to learn more? Dive into our documentation on integrating with Enterspeed.
The Management App have been updated, for this release you will notice some smaller features and changes:
↕️ It is now possible to sort both source entities and views by updated date
⬅️ You can hide the left sidebar to give more space for schemas
📋 The JSON document in the testing pane is saved when opening/closing the testing pane
🌚 Dark mode colours have been given a refresh
⌨️ More keyboard navigation options from the ctrl/cmd-k menu powered by CommandBar
🌍 View connected sources on the Environment settings
We hope you enjoy the small new features and we are looking forward to releasing more Management App features. If you have any feedback or questions, then please reach out on your Slack support channel or via support@enterspeeed.com
Enterspeed has just dropped updates to our Delivery API and Routes API. Both updates support static site generation.
In Delivery API news, we've cranked up the volume on retrieving views. Need a ton in one go? No problemo! With our new POST method, you can now send a batch of handles or viewIds and fetch up to 1,000 views in one request :
The Routes API is stepping up its game and now supports a continuation token for paginating the response. It's not just an update; it's the recommended way to navigate the Routes API.
And as always: Have fun using our API. If you need any help, please make sure to reach out. We're happy to assist!
What happened?
There haven't been any new updates from us at Enterspeed in the last couple of months. There hasn't been anything except the sound of silence.
But fear not; there's no darkness here.
We have been busy optimising and tweaking our processing layer, making it run like a smooth, well-oiled machine.
You see, we experienced an incident with a backlogged processing queue at the beginning of January. This incident caused us to throw everything in our hands and optimise our processing layer with multiple small and large improvements.
This update will show what we've been doing in Enterspeed for the last couple of months and what we have done to avoid similar incidents.
Processing is usually very straightforward: One source entity is ingested, and a transformed view is ready for the Delivery API in a few seconds.
And this works very well. However, at other times, the processing process becomes more complicated.
Examples of more complicated processing scenarios are, e.g. a schema deployment or a re-ingest of all source entities.
Processing in these scenarios results can result in many thousands of processing jobs. Thus processing time is not as instant as in the 1:1 source entity to view scenario.
That is by design, and we don't claim or even want to scale to instant processing times when the volumes become large. So this pre-processing model comes with some latency tradeoffs. But the problem was that one customer could affect other customers, and nobody likes noisy neighbours.
This is, of course, not something we can accept, and we have therefore implemented what we call a "fair processing queue". Without going into too much technical detail, we can now regulate how much processing time is available for different classes of tenants and different types of processing jobs.
Overall, this allows us to remove the noisy neighbour problem so that all tenants will experience that ingesting a source entity will result in a generated view a few seconds later.
The more complicated processing scenarios still create roughly the same number of jobs. We want to underline that the fair processing queue allows multiple customers to share the same resources fairly.
Both from a cost and sustainability perspective, this was the design requirement in the fair processing queue concept we wanted to introduce.
So, we are thrilled with the design and performance of the fair processing queue.
Another issue in those dark hours at the beginning of January was what we internally call "over-processing".
In some cases, updating one source entity resulted in a spike in processing jobs.
One of the key features of Enterspeed is that the developers need to worry less about cache invalidation and leave that to Enterspeed.
In the inner details of these features, we found two bugs that triggered the re-processing of too many source entities. Re-processing too many source entities and a shared queue were the two main issues for the incident in January.
Of course, we fixed the two bugs, but we also added a feature to the new fair processing queue: job de-duplication.
We can now detect duplicate jobs within the same batch of processing jobs to reduce the overall number of jobs.
This saves time and results in a win both for costs and sustainability.
The last thing we want to mention is the small optimisations.
When working with hundreds of thousands of processing jobs - and we saw millions at peak - all the small inefficiencies start to add up to large numbers.
And this is one of the cool parts of working with some really large numbers; the small optimisations make a real difference.
One of the really interesting things was an unexpected behaviour from a specific IoC container registration that caused a spike in CPU usage.
We ended up removing all applicable IoC container registrations of IEnumerable<>, to optimise CPU usage.
We also made several changes to how we query data from our data store.
So if you want to learn more about continuation tokens and point reads in Cosmos DB, don't hesitate to get in touch.
Another tweak to our datastore was to move more of the Cosmos DB containers to tenant-specific containers for our Enterprise tier.
Overall we are very pleased with the changes, optimisations and tweaks we have introduced.
As with all software, we continue learning from our customers - you, our excellent technology partners - as the platform is being used for more than we imagined.
Until next time 👋
Just call us The Pointer Sisters because, boy, are we excited! And no way we’re going to hide it.
This edition of Enterspeed Product Update contains so many incredible things.
So… Let’s get excited!
We teased in our last EPU that the CLI (Command-line interface) would soon see the light of day. That day has finally arrived. We’re proud to announce version 1.0 of the official Enterspeed CLI.
The Enterspeed CLI lets you work with your tenant for all sorts of tasks, for instance:
You can either work with the CLI locally, or you can implement it in your CI/CD pipeline – whatever fits your workflow.
Head over to the CLI documentation to see how you can get started using it.
You can also check out this introduction video that our colleague Jesper W. made.
We love coding. But sometimes less is more, which is why we base our schemas on low-code principles.
Often you want to get something done quickly and efficiently without manually having to type a lot of code.
Well, we got your back, Jack. We’ve made it even easier to create your next schemas thanks to our new schema snippets feature.
Simply start typing the name of the field, property type, or example schema you want to use, and a list of options will appear in the editor. Click the tab button and BAM! You got yourself some code 👨💻
How easy is that?
We currently support the following shortcuts for schema snippets:
Do you know what can be a pain in the neck to keep track of? Different schema versions deployed to different environments independently of each other.
That is why we created a “Versions” overview page. It lets you see exactly which version of your schema is deployed to which environment.
Neat 📸
Well, now you no longer have to (even though we can understand if you still want to be like Travolta, he’s got some sweet dance moves) thanks to our “schema compare versions” functionality.
On your schema page, simply select the version you want to compare with, and a “diff checker” will appear, highlighting all the changes from the previous version.
You can even edit your current version at the same time.
Our last few integrations have been focused on traditional CMS, but now it’s time to dive into a new category. This time with one of the worlds leading commerce solutions, commercetools.
commercetools, founded in 2006, is a cloud-based headless commerce platform with customers around the world.
We’re so happy that we're now able to support these customers with this integration.
The integration lets you import your product, variants, and categories into Enterspeed.
GitHub link: https://github.com/enterspeedhq/enterspeed-source-commercetools
We have also developed a new integration to Contentful, the cloud-based CMS. With this integration, you can ingest your content from Contentful into Enterspeed.
The integration is now in beta and does currently not support all events and data types in Contentful.
GitHub link: https://github.com/enterspeedhq/enterspeed-source-contentful-cp
One of our most popular integrations, our Umbraco integration, has just had a major update. To make things easier for users (and for us), we have unified our different Umbraco integrations into one, starting from Umbraco 9.
This makes it easier to update for the users, easier for us to maintain, and much easier for everyone to navigate the documentation.
The new unified integration supports Umbraco 9, 10, and 11, as well as all future Umbraco versions.
We have also added support for the new Block Grid Editor for Umbraco version 10.4 and version 11.
You can read more about the integration right here: https://docs.enterspeed.com/integrations/umbraco/getting-started
We have made quite a few demos and articles based on exciting frameworks like Next, Nuxt, Astro, you name it.
But sometimes you may not feel like having the Chocolate chip cookie dough ice cream with sprinkles and whipped cream on top. Sometimes you just feel like… Vanilla.
That is why we’ve created an unopinionated “Getting started” tutorial, completely free of any CMS or framework and simply based on good old “vanilla” JavaScript.
You can find the tutorial right here: https://docs.enterspeed.com/tutorials/creating-a-simple-site/intro
This is the good old “Have you tried turning it on and off again?” – but for schemas.
Sometimes stuff happens. For one reason or another, you might need to generate a new view without needing to update your source entities.
This can easily be done by creating a new version of your schema and deploying it. But that quickly gets annoying if you need to do it multiple times – and you risk ending up with a lot of versions of your schema.
We have introduced an easier way to do this. In the bulk deploy pane, you now have the option to redeploy already deployed schemas, meaning the version will stay the same but will trigger a regeneration of your view.
In the last edition of EPU, we introduced schema folders, and boy, where you guys existed. That's until you realised that you had to prepend the folder name in the schema name in order to place it in the folder.
Then suddenly, it felt like being back in 2012 with some of you saying, “Ain’t nobody got time for that!”.
Don’t worry, we get it, which is why we've made a shortcut button for creating schemas directly inside your folders.
This last improvement is yet another benefit for us lazy people. Yay!
When creating a schema, we previously introduced the option to prefill your source in the schema. Now we have taken it one step further by also enabling you to select source entity types.
These source entity types will automatically be added to your schema’s triggers inside the data source.
• Source entities filters are now saved when navigating to a single entity and back.
• Umbraco integration: Fixed that media domain was not added to all media types.
• Improved visibility of dropdown labels in dark mode.
Until next time 👋
In this Enterspeed Product Update, we got a lot of schema-related goodies for you. Not only is it now easier to quickly create a schema – but we have also introduced a new way to keep them organised.
And as always, we got a lot more up our sleeve – so let’s dive in.
When building large sites, you often need to create a lot of specific schemas. As the number of schemas grows, so does the complexity.
You guys started asking for a better way to organise and keep track of your schemas. And just like a mad scientist we grabbed our lab coat (hoodie) and headed straight to the lab (VS code).
The result is schema folders. Instead of having all your schemas in one long table, you can now organise them into folders.
Simply add a slash (/) in your schema name and the text before will be transformed into a folder – pretty neat!
To keep it easy and manageable, we've limited the number of folder levels to two. This means you can organise your schemas into folders and subfolders.
For instance, a schema with the name “Global/Various/Link” will be transformed into a schema with the name “Link”, located in the subfolder “Various”, which is located in the folder “Global”.
Neat! To quote old Benjamin Franklin: “A place for everything, everything in its place”.
Now it’s quicker and easier to create a schema. When creating a new schema, you now have the option to select a source.
Selecting a source will automatically prefill your schema with the available source entity types. We've also created a “dynamic mapping”-property, which lists all the data available for the source entity. That way it’s easier to quickly test your sources.
When creating a reference, you are now able to define a different source. You do this by adding the “source”-property and making it equal to the desired source group alias.
By that, you now can use a view that isn't in the same source group as the rest of the schema.
Please note, that the “source”-property only can be used if “originId” is used. If “id” is used the source of the id will take priority over the source.
We released a new version of our Delivery API – version 2. The new version now supports multiple handles for the same schema.
In version 1 the schema alias is used as the key for a handle. Because of that, it’s not possible to fetch multiple handles of views generated by the same schema.
In version 2, however, the key is changed to the handle name, meaning that you can fetch multiple handles even though they are generated by the same schema.
We’ve also simplified the API and made it "cleaner". First, the key is the same as the handle name you request and not a schema alias you may or may not know about.
Second, we have removed the id, type, and view property and made the view data inline for view references – making it much cleaner for the client.
Last but not least, we’ve added the "missingViewReferences" in the meta field, which makes it easy for you to debug missing view references. All missing view references get listed here with the id of the missing view and a path of where the view is missing.
When selecting a source on the Source entities page, Generated views page, etc. you can now see which source group each source belongs to.
Thus, you can easily distinguish between similarly named sources.
Need to duplicate a partial schema? No problem.
You can now easily duplicate your partial schema, just like you can your regular schema. Select “duplicate” in the settings menu, which you’ll find in the partial schemas overview.
Friends don’t let friends accidentally delete an environment in use!
And at Enterspeed, we’re most definitely your friend, so we’ve made sure this won't happen on our watch.
As a security, environments that are referenced from a data source or environment client can no longer be deleted.
If you need to delete an environment, you will first need to unlink it from your data source and/or environment client.
Ever tried dealing with noisy neighbours? Yeah, not much fun. You might have thought: “I would pay anything to have some peace and quiet.“
If you go Enterprise with Enterspeed, then you can. We've taken the first steps towards providing dedicated resources for our Enterprise plan customers.
As both we and our customers grow, a predictable processing speed is important for us.
By dedicating a slice of our processing layer to individual enterprise customers, we've effectively eliminated noisy neighbours for them.
…We have also added a bunch of features and fixes to our Umbraco Source connector – head over to the changelog to view more!
A highly requested feature will soon see the light of day – the Enterspeed CLI.
Our skilled engineers have been working tirelessly on the first version of the Enterspeed Command Line Interface.
We’re really excited to share this with you all. The CLI will bring a whole new way of working with Enterspeed, by having it available via your favourite IDE.
Soon we will be letting the first few customers try out our beta, so we can get some feedback before releasing it.
If you’re interested in testing it out – feel free to drop us an email at info@enterspeed.com
Until next time 👋
You can say that Christmas has come early for all you Umbraco-users out there – and just like Santa’s bag, this product update is filled with existing things.
And don’t worry, unlike Santa we won’t ask if you’ve been naughty or nice.
We’ve added yet another integration to our stack – the Umbraco V10 integration.
Umbraco V10 is the latest version of Umbraco and brings a lot of existing features, for instance, SQLite support, .NET 6 and ASP.NET Core 6, Nullable reference types, and much more.
The integration is available on GitHub and via NuGet. You can find more information about the integration in our documentation.
Documentation: https://docs.enterspeed.com/integrations/umbraco/umbraco-v10/getting-started
GitHub: https://github.com/enterspeedhq/enterspeed-source-umbraco-cms
NuGet: https://www.nuget.org/packages/Enterspeed.Source.UmbracoCms.v10/
Now you’re maybe thinking “Aw, snap! I’m only using Umbraco V8 – I wish there were an easy way to upgrade”. Well then, get ready to get your mind blown 🤯
Imagine this. You are using an older version of your CMS, this could be Umbraco V8, and you want to update to the latest version (in this instance Umbraco V10).
How do you do this easily and painlessly, without it affecting your live website? You use Enterspeed and our latest extension, our Enterspeed Migrator.
The Enterspeed Migrator currently works in Umbraco V10 and lets you copy all your templates and content into your Umbraco V10 instance.
The way it works is by fetching the source entities ingested into Enterspeed and then building your templates and content in Umbraco.
Look at this video of how we migrated from Umbraco V7 to Umbraco V10 in no time: https://www.youtube.com/watch?v=8DOUJpbc6CM
That’s pretty wild. Do you know what’s even wilder? We also did this from a Sitecore site to a new Umbraco site. That’s right. We changed the freaking CMS, which was almost as easy as changing your bed linen 🛌
Imagine the look on your manager's face, when you do a migration that usually takes days (depending on the size of the site), in the time it takes you to drink a cup of coffee.
Insane.
You can find the Enterspeed Migrator here: https://github.com/enterspeedhq/enterspeed-migrator
Our Umbraco integration now also supports Umbraco Media, meaning source entities will automatically be created in Enterspeed whenever you save a media item.
If you save a folder in Media, all its content will also be ingested to Enterspeed. Using the “Seed to Enterspeed”-functionality also ingests all Media items.
This makes it possible to access the media’s meta information, for instance, width, height, type, etc.
Our Umbraco integration already includes a default property service, that maps all properties (including some meta-data) that are present on all data sources.
Some might need to extend the meta-data, for instance, to include site settings or domain name data. For this, we have created a Custom Property Service, which makes it easy to extend the meta-data on sources.
You can find more information about this in our documentation.
...that was some sweet Umbraco updates 🙌
You may be setting with a slight frown on your face thinking “What about me? I don’t use Umbraco – doesn’t Santa have anything for me?”.
Don’t worry, kiddo, of course, we didn’t forget about you 🎁
That’s right, you can put your sunglasses away – we’ve added Dark mode support in the Enterspeed app.
Dark mode is properly one of the most requested features in software – and we get it. Once you have switched your IDE from the default light theme to a dark theme, there’s no going back.
💡 Tip: Our code editor inside the Enterspeed app supports themes, meaning you can keep the light theme and include a dark-mode editor. You’ll find the theme switcher at the bottom of the editor.
Instead of having multiple different repos for our demos, we have now moved them all to a single monorepo.
While doing this, we have also added demo content to each individual demo folder, making it easier to find and use.
We now have 11 different demos showing how to use Enterspeed:
You can find our demo repo on Github: https://github.com/enterspeedhq/enterspeed-demos
Until next time 👋
We know, we know. It's been a long, long time 🎵
But don't worry, unlike the late Kitty Kallen, we won't ask you to kiss us once nor twice.
Instead, we'll introduce you to one of our biggest updates yet. You see, we haven't just been slacking off this whole time. No sir/ma'am, we have been busy working on our new bulk schema deployment, among many other things.
But before we can talk about this, we need to dive into our new source group concept - because, just like string theory, it's all connected 🧵.
We have introduced a new layer to the data sources called source groups. Adding the source group concept was a necessity to make our new bulk deploy flow as simple to use as possible.
A source group wraps around your data sources and bundles them into a single group. This helps you to keep better track of your sources.
Most developers work with multiple environments (meaning multiple data sources) when developing a project, for example, Production, Staging, and Development.
Having all these data sources grouped together makes it easier to manage, and helps avoid mistakes, for instance, accidentally adding one source to multiple environments.
Source groups now also make it possible to use actions across multiple sources 👏.
When using source groups, the new schema field, triggers, will be replacing the old sourceEntityTypes schema field.
Alas, it's time to say goodbye to the schema field sourceEntityTypes.
When using the new source groups, it's no longer possible to use sourceEntityTypes. It will now be replaced with the new schema field: triggers.
The triggers-field is an object containing your source group(s) that contains an array of source entity types.
Enough beating around the bush. Let's get into the main star of the show - the new bulk deploy feature 🌟.
Deploying changes, keeping track of changes, and mirroring environments just got ten times easier.
Instead of having to go through multiple schemas and releasing one by one, you are now able to do it in one go.
Not only does it save you time - but it also minimizes the risk of errors, since you get a complete overview of the current versions and versions which are going to be deployed.
Moreover, since schemas can be interconnected, the option to deploy simultaneously reduces risks even further.
Instead of having a deploy button on the "single schema view" a "deploy changes"-button has now been added to the header.
Clicking this opens a pane, which allows you to select which environment you wish to deploy your changes to.
A list of all schema changes is shown, alongside the currently deployed version, which version will be deployed, and the state of the schema (added, updated, deleted, unchanged).
You can also choose to exclude a schema from deployment, deploy another version or simply unpublish it.
If you wish to unpublish or change a version of a schema that doesn't have any current changes, you can click "Load unchanged versions" to append the table with the rest of your schemas.
Last but certainly not least, a much-requested feature. You now have the option to copy versions from another environment - meaning mirroring an environment.
This is really useful when testing things out on the Development environment, which you wish to copy to the Production environment.
To do this, simply select the environment you wish to use the versions from in the "Use versions from different environment"-select and click apply.
Afterward, click Deploy X changes and confirm. After the schemas have been validated they are now deployed to your environment.
Now that your head is almost filled to the limit with pure awesomeness, allow us to introduce a feature that will make your head explode.
One of the powerful things about Enterspeed is the option to model and transform your content however you like.
However, there are times when your current content model matches your needs, and transformation isn't necessary.
This is where dynamic mapping shines, which is done by using our dynamic property type.
Say you have a product where you want all of its properties (title, image, description, price, stock, etc.). Instead of mapping all of these single properties out, you simply type:
"product": {
"*": "p"
}
Now all properties from your source entity can be accessed using p.thePropertyName.
Say you only want the features? No problem, you just define this by appending "p." with the property name like so:
"features": {
"*": "p.features"
}
You can read more about how it works in our documentation under Path selector.
We have expanded our integrations list with two new integrations for Sitecore - version 8 and version 9.
These integrations let you automatically push data from your Sitecore solution into your Enterspeed-project.
Furthermore, we have added in-depth tutorials for both integrations, which you can find in our documentation under integrations.
Just like googling is a core skill for all developers, so is copy-pasting code snippets (thank you, Stack Overflow).
We're all on board with this, which is why we have created a schema snippets section in our documentation.
Happy copy-pasting🙌
We use the Monico editor for our schema editor. If it feels familiar, it may be because it's the same editor that powers VS Code.
This also means it comes built-in with a ton of handful features - including the command palette.
To run a command, simply press F1 or click the "play"-button on the right side of the editor to open the menu.
We have added a "band" to the bottom of the schema editor. We have moved Save-status, Autosave-option, Theme-selection, and the WordWrap-option to this "band".
All schema errors will also be shown here, in the bottom left corner.
When doing a "dry run" (testing your schemas on source entities), we have now included schema warnings, alongside schema errors.
This can help you spot redundant code, for instance, referenced source groups that are no longer in use.
Until next time 👋