Introduction
Machine learning is becoming more popular every day. In one of our previous articles we talked about dialogue systems and showed you how simple it is to make an AI application using some machine learning frameworks. In this article we will review the top 5 most popular deep learning frameworks.
Why just deep learning? First, this extremely interesting theme is becoming more powerful and more important in the data science sphere. Second, deep learning libraries come out on top in almost all machine learning projects. Yes, we can talk about Apache Machine learning infrastructure or The Microsoft Cognitive Toolkit, but we think that it’s much more interesting to talk about tools that you use yourself. Let’s start our review!
TensorFlow
This framework is developed by Google and was initially intended for the calculation of differentials and tensor calculus. But time goes on, its popularity grew, and this open source library began to be used in machine learning. So, it’s a very low level numerical library, but it provides almost unlimited possibilities for designing your experiments. If you have strong math knowledge and understand how neural networks work “under the hood”, you won’t have any problems using TensorFlow.
TensorFlow has a cool visualization system named TensorBoard, where you can watch some graphics, computational graphs, and fitting processes.
Of course, some people (especially beginners) get frightened when they read code written on TensorFlow. But for these people, the next framework, named Keras, was created.
Keras
Keras is a good “frontend” for TensorFlow (or Theano, but this framework is losing its popularity). This framework helps to design your neural networks quickly and in a more readable format. It’s a good, powerful instrument to start your learning or experiments, but it’s very expensive to use in production: Keras has some problems with memory leaks.
PyTorch
One of the most powerful and perhaps the fastest growing deep learning framework. It’s the main competitor to TensorFlow. It is developed by Facebook, and is a pythonic version by Torch. It’s the other deep learning framework that is written in Lua (an exotic scriptable language).
PyTorch follows the object-oriented paradigm and so it has more readable and clean code, like TensorFlow. It’s a very pleasant framework and many companies like Facebook and Twitter use it in their research.
Caffe2
Caffe is a very popular and widely used computer vision library that ported Matlab’s implementation of neural networks to C and C++. Caffe2 is the second version of this framework. Like the first version, it uses Python API with a C++ engine and is used in large-scale and resource-dependent projects. Caffe2 is mainly used for production environments in big companies (like Facebook and Amazon).
Mxnet
It’s a very interesting machine learning framework that has an API for Python, R, Julia and even JavaScript! Mxnet was created especially for integration with Amazon Web Services, but this flexible framework became popular not only among users of AWS. One of the maintainers is a team of researchers at the University of Washington.
Other
Nowadays we have many other frameworks, like Microsoft CNTK, h2o.ai, deeplearning4j, etc. Besides these, we shouldn’t forget classical frameworks like python machine learning framework scikit-learn, Apache MLib or Apache Mahout. All these technologies are used very actively in projects, but some frameworks like TensorFlow and PyTorch evolve with cosmic speed! Deep learning is catching the interest of more and more engineers. And the future will be focused on neural networks. But don’t forget that classical machine learning always will be necessary.
How to choose from many machine learning frameworks?
To start, you should answer some questions.
What is your project’s infrastructure?
Of course, now almost all frameworks are platform-independent, but sometimes this causes some problems. For example, if all your components use Apache stack (Zeppelin, kafka), it’s better if you use Apache Mahout or Apache Singa. It will make your project’s ecosystem more flexible.
What is the scale of your project?
Yeah, this is such an important question. Your choice should be customer-oriented and scalable. If you develop a small product or your product is intended for internal use only, then you can use Tensorflow or PyTorch. In your production version, it will be safe and fast. The same situation can happen if your machine learning models don’t refit every second and manipulate fixed data.
But if you want to integrate some deep learning frameworks into your large-scale project, you should consider Python’s low speed and productivity (Tensorflow or PyTorch are the most popular frameworks directly for Python). And Caffe2 comes to the rescue! Yeah, this is no laughing matter! It’s well-known information that Facebook software engineers make their research prototypes using Pytorch. And after that, they save the network structure, save their code and rewrite this research to C++ using Caffe2. But notice: it is a very hard process. You can’t afford this if you work in a small startup. If you don’t have the resources that Facebook, Google, and Amazon do, every new model implementation will eat into your time. Remember that if you want to rewrite something using Caffe2, you need to allocate more time to the project or look for developers carefully. By the way, what about the necessary qualifications for developers?
How to find the right team?
Nowadays we have a huge amount of different information about machine learning frameworks. Tutorials, good documentation, articles, and reports from conferences are among them. And every serious and strong software engineer should be very inquisitive.
Of course, it’s impossible to know everything. But if you are looking for developers, you should be confident that they are hungry for knowledge. If you want to use TensorFlow in your project, but your potential developer knows only PyTorch, by and large it doesn’t matter. Because a good specialist should understand internals. Some fields that should be covered by your team:
- Linear Algebra
- Strong knowledge of programming language (Python, R, or C++)
- Statistics
- Mathematical analysis
- And of course, Machine learning, Data Science and Deep Learning fields
It simplifies the choosing a framework problem if your software engineers have some expertise in this. But a good team can’t do anything without powerful hardware. Note that you have to take care of this.
What hardware do you need?
This is a very important question. If you want to develop a cool popular startup with deep learning, you should take care of this in advance. Of course, two Nvidia GTX 1080 TI video cards are enough to win the Kaggle competition. But for 24/7 production work this is not enough. You can rent servers using AWS, Google Cloud, Microsoft Azure or other services. But don’t forget that “cool deep learning” and “popular neural networks” are not just beautiful words. They require a lot of GPU’s, time, electricity and mystery. Be ready for it.
Conclusion
In conclusion, I want to say that to choose the best machine learning library and framework it is important considering not only personal preferences, but also the current project infrastructure. Modern framework leaders are developing at great speed, and it’s hard to choose one. But if you are familiar with Python we suggest you take a closer look at PyTorch. It’s cool for diving into the deep learning magic. If you have your own idea for machine learning project, don’t hesitate to contact us and ask any questions. Let’s put your ideas into practice together.
By
Oleg Chislov
Data Scientist
@noeliagorod 