How to create your first large scale AIAAS?

https://miro.medium.com/max/1200/0*XZqoMqeC4IKtL_yo

Original Source Here

How to create your first large scale AIAAS?

2022 is the Artificial Intelligence As A Service’s year (AIAAS), and many interesting services are still to come. However, developping AIAAS is still difficult because many Artificial Intelligence models require lot of memory or computation power, and very often both of them. Let’s see how to cope against those challenges, even for services at a large public scale.

Photo by Annie Spratt on Unsplash

Cloud services

First things first, the cloud services. Currently, the 3 main cloud services from Google, Amazon and Microsoft have very similar functions. That’s why it is difficult to say which one is the best in general.

Here is an overview of the 3 main cloud services, with some noticeable advantages.

Main cloud services. Image by author.

For an AIAAS, 1 or 2 months of credit or free trial is very short, because the development cycles between your AI model and the other services are longer than classic ones. Consequently, if you want to reduce costs, it is highly recommended to spend more time preparing the model and the services locally, before implementing the AIAAS in the cloud. Another good idea is to start working with 2 or 3 cloud services to see which one fits your needs the best.

All the 3 cloud services have plenty of servers accros the world, which is important to make your AIAAS available very fast and anywhere.

There are also plenty other cloud companies with interesting prices and services, but they might not be suited for large scale AIAAS deployment.

In all the cases, it is crucial to be very careful about the contract terms and the service cost. Many users have lost plenty of money due to a wrong evaluation of the service costs.

Cloud costs

Classic SAAS use libraries that doesn’t require lots of hardware resources. This is far from being the case with AIAAS: AI models are between a few megabytes to several gigabytes, and applying them at a large scale can have a huge cost.

Remember that models must be processed by libraries that take generally a lot of disk space. For instance, Tensorflow takes about 1Gb of disk space, and in many cases more than twice in RAM space.

Without to mention that cloud services use emulators like Unicorn that increase even more disk space and RAM. That’s why the deployment in the cloud will differ from a local deployment, due to a very different infrastructure.

However, cloud services can auto-scale the needed hardware resources regarding incoming new customers and hence make AIAAS possible. Autoscaling means that they add or remove servers’ instances according to the current number of visitors.

Cloud services propose many hardware setups, from low memory/calculation power to high. Depending on the service model, you have to fit it to the hardware as best as possible.

Cloud service average cost per model size and per instance. Image by author.

The previous table shows the cost per month per instance and it should be evaluated very carefully. Having an instance costing 500$ per month is not expensive because it could cover 10000 visits or more, and generate 1000$ sales with 1% of those 10000 visits.

In addition to that, there are many solutions to reduce the memory, the training time and the calculations necessary to run your model like pruning, freezing or lighter structures like tensorflow-lite or ONNX. It might not be enough in many cases to have very fast AIAAS, that’s why you have to deal with a specific AIAAS constraint: time.

Photo by Hannah Busing on Unsplash

Training times

Depending on the data science process you are dealing with, training times could be more or less long.

It is mostly dependent on the final model size and hardware, but it is also linked to the model complexity and the data type (image, text, sound, etc.).

Here is a simple comparison between 3 machines. It is not covering every scenario (NLP, image classification, etc.), it is just to give a rough order of magnitude.

3 main hardware config for AIAAS in 2022. Image by author.

The following table presents some duration values compared to model sizes.

Rough Order of Magnitude of training and processing time using the 3 harware config. Image by author.

Some models could be huge like GPT-3 and need powerful hardware like Nvidia’s A100. There are interesting things to know in terms of energy consumptions, costs and alternatives described in this article.

The other hardware setups are commonly used in the market or in cloud platforms.

Note: As I didn’t find any clear comparison between GPUs and CPUs in terms of training and processing time, I’ve done an overview based on several pieces of articles and my personal experience. It could require some adjustments, so please feel free to give feedback to improve the values.

Processing times

Classic webservices must answer in less than one second, otherwise most users quit.

That’s why AI processes, which are longer than classic ones , could be seen as a handicap for a service.

Nevertheless, if AI is still faster than humans, people would wait if the benefit worth the patience. That’s why it is necessary to show why the process takes several seconds thanks to a clear explanation (for example with a loading animated gif), or, if it is really long, thanks to an on demand service, which result will be sent in a few minutes or hours. Otherwise, cloud services have fast GPUs available to do very fast calculations, but it would increase the overall cost. Be careful because cost might explose because of heavy hardware setups. Cloud cost simulators are here for that. Up to you to find the right balance.

Photo by Patrick Ward on Unsplash

In addition to that, there is other times to take into account: the model installing and loading times. It happens when you deploy a service or add new instances automatically or not. This could be a problem when you have to deal with larger models that takes several minutes to install or load. In such case, fast installation using SSD might be interesting.

Hardware for development

Despite the fact that everything could run on a webservice, we cannot ignore that most devices used in the world are on Android and iOS, and if our AIAAS is not running on smartphones, we will lose an important market share, even if the webservice is adapted to smartphones. The fact is that most people prefer having their services in apps rather than in a web window.

Unfortunatelly, even if Mac has an optimized hardware architecture, it is not the best option to train models without a fast GPU. On the other hand, PCs cannot be used for iOS development due to Mac policy.

That’s why the cheapest solution to cover 100% of the device market, is to use a recent but affordable Mac to develop iOS apps with XCode, together with a ML cloud service like Paperspace to train models with GPUs.

Note: Even multi-platform languages such as Kotlin or Flutter require a Mac to develop iOS apps.

Harware cost to cover 100%of the global market. Image by author.

Keep in mind that there are deep learning libraries adapted to iOS and Android smartphones, but if the model is too large, you can access it remotely from an app with a secured web socket.

Security

AIAAS requires a higher security than classic SAAS, due to the high processing cost that could be stolen, using hacking actions to get free requests. That’s why Recapcha, CSRF protection and all cloud security rules are mandatory.

It is highly recommended to use secured cloud services such as GCloud, Azure or AWS: they will not protect you from all kind of frauds, but they will at least secure all the inner connections between the data base, the model and the interface.

Photo by Mika Baumeister on Unsplash

Profitability

Ads could be interesting in some cases, but the numbers of view must be very high to make the service profitable. That’s why ads are usually not suitable for AIAAS, as the high processing cost cannot be compensated by them.

Like many modern SAAS, the best option is often a free few tries, and an propose to a monthly or yearly subscription. In general, about 1% or 2% of visitors would pay a monthly subscription, depending on the service attractiveness.

Most models can be tuned with parameters and could add value to the service. Making the service profitable means also finding interesting ways to tune the model and hence increase the customers’ curiosity.

Marketing

There are several modern ways to announce your AIAAS:

  • Automated chatbot in social networks to answer specific people who might be interested, overall if the service is based on NLP.
  • Most newspapers are willing to publish latest technology trends, and any new AIAAS is welcome.
  • Sell derivated products through a marketplace like Zazzle or Redbubble. You don’t need to manufacture anything : just upload your designs and sell.
  • Social media influencers.

Conclusion

AIAAS is now possible thanks to cloud platforms and powerful hardware, but it requires a very good business model to be profitable enough. That’s why, a constant model improvement is necessary, in addition to creativity by using model parameters or innovative marketing approach.

AIAAS being a complex topic, please feel free to send feedback to improve the content.

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: