Three years ago, AI became a new selling point for smartphones. For a time, smartphones supporting AI features quickly spread, and AI performance has become the most important mobile processor parameter after the CPU and GPU. Today, demonstrating AI performance has become an indispensable part of mobile phone and mobile processor conferences.
But have you ever wondered why you need an AI-powered phone? What exactly can an AI processor in a phone do? Reasons you need a phone with AI processors
When it comes to the AI function on mobile phones, many people can immediately think of it-smart voice assistant and AI camera. There is no doubt that AI photography is the most commonly used function by ordinary consumers. Features include AI background blur, AI super night scene, AI beauty, AI noise reduction, etc. In addition, some common AI functions include face recognition, call noise reduction, translation, etc.
Right now, these common mobile AI features are becoming powerful. For example, in the past, the functions of automatic speech recognition (ASR) and offline real-time speech translation could not be implemented on mobile phones. With the advancement of AI technology and the deepening of cooperation between hardware and software companies, ASR and offline real-time speech translation can be used on mobile phones. It can even respond in noisy environments.
In addition, in terms of taking pictures, the mobile phone camera has been increased to three or even five. During the process of switching between wide-angle, ultra-wide-angle, and telephoto lenses, users can easily feel the freeze, while Qualcomm and its partners jointly completed the optical The zoom smooth switching solution can bring users a smooth zooming experience based on AI.
Smooth switching of optical zoom based on AI
Compared with smart voice assistants and taking pictures, the AI functions on some phones are not easy to find. For example, the short video platform of the world is very popular. For the consideration of network bandwidth, the user ’s mobile phone actually receives 320P resolution video, but higher resolution means better experience. For this reason, In-depth cooperation between Byte Beat and Qualcomm, optimized based on Snapdragon 865, and using a mobile phone’s AI processor to run a special neural network (super-resolution convolutional neural network), the video received by the user is 320P resolution per frame Upgrading to 720P, because this process is implemented through terminal-side AI, users can also get a smoother experience.
Not only that, AI can also be used to solve the battery problem that makes many users anxious. The iPhone uses AI capabilities to learn the user’s habits, thereby optimizing charging and extending the battery life. Qualcomm also introduced the AI algorithm into the entire system of fast charging and discharging in its Quick Charge technology. By learning the user’s personalized usage habits, such as playing games and the duration of the game, and then training, Quick Charge AI can be very good The entire system’s CPU includes charging efficiency.
In this way, Quick Charge AI can further improve the prediction accuracy of the remaining operating time of the mobile phone battery, up to 15%. In addition, it can achieve better charging heat management, extend the battery’s cycle charging life, and can extend up to 200 days.
Of course, AI also helps mobile games, shopping, learning and other functions become more interesting and practical.
In fact, the powerful AI capabilities of mobile phones require powerful mobile phone processors and AI capabilities to support them. With the recent release of the Snapdragon 865 mobile platform, let’s analyze how the hardware and software cooperate to achieve powerful AI functions.
Higher-performance AI processors, more powerful AI functions
Earlier last month, Qualcomm released the latest flagship mobile platform Snapdragon 865. The new platform integrates 5G and AI well, can provide peak speeds up to 7.5 Gbps, and is equipped with the fifth-generation Qualcomm artificial intelligence engine AI Engine and the sensor hub (Sensing Hub ). In addition, the Spectra 480 ISP can achieve processing speeds of up to 2 billion pixels per second. Snapdragon Elite Gaming can also support a series of new features such as end-game experience and extreme realistic graphics performance, which can bring users excellent shooting and gaming experience. .
As the focus of the Snapdragon 865 upgrade, the performance of the new Kryo 585 CPU has increased by 25%, and the overall performance of the new Adreno 650 GPU has been improved by 25% compared to the previous generation platform. The AI Engine most closely related to AI functions, the fifth-generation AI Engine integrated by the Snapdragon 865, has a performance of up to 15 trillion operations per second (15 TOPS), which is twice the performance of the previous generation Snapdragon 855 AI. Compared to the Snapdragon 845 AI, the performance is improved by 5 times.
The significant improvement in Snapdragon 865 AI performance does not depend on the improvement of a certain processor. The AI Engine has always adopted a multi-core heterogeneous computing solution. Through the collaborative processing of the Kryo CPU, Adreno GPU, and Hexagon processor, the AI task is jointly completed. The fifth-generation AI Engine core Hexagon processor has been completely upgraded. The TOPS performance is 4 times that of the previous-generation tensor accelerator, and the running energy efficiency is improved by 35%. The AI computing power of Snapdragon 865’s Adreno 650 GPU has also increased by more than two times compared to the previous generation Snapdragon 855 GPU AI computing power, making it the strongest mobile AI processor.
Of course, the AI Engine multi-core heterogeneous architecture can also achieve the best balance between performance and power consumption. For battery-powered smartphones, it is important to have both high performance and low power consumption. Like the idea of improving the performance of AI Engine, Qualcomm’s reduction of power consumption is not limited to a certain hardware, but from a system perspective. optimize.
Deep learning algorithms are currently the most commonly used algorithms in AI. Such algorithms bring a lot of convolution operations, but the most energy-consuming is not the convolution calculations, but the data handling. For this reason, Qualcomm upgraded the bandwidth compression technology used on the Snapdragon 820 to the deep learning bandwidth compression technology on the Snapdragon 865 AI Engine, which can achieve lossless compression of up to 50% compression ratio. Access to memory. In addition, the newly supported LPDDR5 of the Snapdragon 865 can bring about a 30% increase in bandwidth and jointly reduce power consumption.
It should also be mentioned that in order to further enhance the intelligence of the mobile phone processor, the Snapdragon 865 also integrates a sensor hub, which can be used for real-time perception of audio and video. In terms of power consumption control, the multi-keyword language voice wake-up power consumption of this sensor hub is less than 1 milliamp. With AI Engine, the mobile phone can sense the surrounding environment with very low power consumption.
For example, the ASR mentioned earlier, Qualcomm and Google have jointly enhanced the Android Neural Networks API, and migrated the Google Assistant ’s speech recognition function from the CPU to Hexagon, reducing power consumption by 3 times and reducing latency by 30%. Speech translation to further improve situational awareness AI to a whole new level.
However, even the most powerful hardware requires software to unleash its “magic”.
More interesting AI features are coming
Apps such as Douyin, Snapchat, Google Translate, Youdao Translate output the mobile phone’s AI functions, which is also the result of close cooperation between mobile phone AI software and hardware developers. However, there is a huge gap in the cooperation between the two parties. On the one hand, software developers do not understand the underlying hardware, and calling and optimizing hardware is a huge problem for them; on the other hand, hardware providers often have a variety of software developers. Not enough understanding of the needs of the development, it is difficult to meet the needs of all developers.
Therefore, unlike traditional chips, the providers of AI chips need to provide a complete solution covering hardware, software, and tools to ensure that everything works together. From the top to the bottom are applications, frameworks, runtimes, libraries, and hardware accelerators.
As far as frameworks are concerned, AI developers may use frameworks such as TensorFlow and PyTorch. AI chip companies will try to support mainstream AI frameworks. To simplify the work of AI developers, the Snapdragon 865 currently supports more than 160 operators. Operators refer to complex functions that exist in the framework and can help tensors work better. It is also worth mentioning that developers can also use Adreno-supported Open CL and Hexagon SDK to create custom operators, so that developers can achieve differentiation on the framework, and can also make full use of the powerful AI algorithms provided by the Snapdragon platform. force.
In the Runtime layer, in order to support AI performance on mobile terminals, Qualcomm is the first company in the world to launch an AI software toolkit for mobile platforms. The collaboration between the Qualcomm Neural Processing SDK and the Android Neural Networks API can provide non-parallel developer access for first-party and third-party applications on the Snapdragon mobile platform, and better integrate AI hardware and software.
It is reported that Qualcomm’s neural processing SDK focuses on improving energy consumption, performance, access and other aspects. It also provides monthly version updates, and works closely with partners, especially OEMs, to provide top AI solutions to support more. Network and higher performance. The Qualcomm AI research and development team also worked closely with Google to optimize the Android Neural Networks API, achieving a 3-5 times performance improvement and making access easier.
What needs to be added is that Qualcomm also launched Hexagon NN Direct. For example, through cooperation with Google, developers of TensorFlow Lite can directly access the terminal beyond the runtime, and libraries that run directly on the Hexagon core can also provide the same for other solution providers Access.
This feature can bring a significant improvement. Snapchat can improve the video from 10 frames per second to more than 40 frames per second by using Hexagon NN Direct.
In addition, Qualcomm also launched a new AI model efficiency toolkit. This toolkit can reduce the model’s redundancy by 1%, and achieve 3 times the model compression with a loss of accuracy of less than 1%. Most use cases are more than enough. In addition, this toolkit can compress 32-bit AI models to 8-bits, increasing performance per watt by more than 4 times.
Based on high-performance hardware and easy-to-use software, Snapdragon’s AI technology has empowered more than 1 billion terminals worldwide. With the further improvement of hardware performance and the iteration and innovation of software, more and more AI applications in XR, work, social and other fields will be available.
Mobile phones were the first intelligent devices to be popularise with AI technology, but two years ago, consumers had very limited AI capabilities. With the continuous improvement of mobile phone AI performance and the continuous iteration of AI algorithms and software, AI photography has become the most popular AI function for consumers in 2019.Because of the powerful AI performance of mobile phones, it was previously only possible to rely on the cloud The implemented AI tasks can also achieve good results in the terminal. At the same time that mobile phone AI functions are more practical, it has also spawned many more interesting and novel AI functions.
Qualcomm, the market leader in this market, has the latest flagship mobile platform Snapdragon 865, which has excellent AI performance and also benefits from long-term accumulation. As early as 2007, Qualcomm Research launched the first artificial intelligence project. Since then, it has continuously strengthened research and development in the field of AI, and established Qualcomm AI Research in 2018.
Of course, the success of the chip, especially the success of the AI chip, the better integration of software and hardware, and the prosperity of the ecology are more important. With strong technical strength and appeal, Qualcomm already has many AI partners around the world. The performance of the field is therefore worth looking forward to. Especially Qualcomm, which is also leading in the 5G field, the fusion of 5G and AI will collide with unexpected surprises.