"If you see the Tao and forget the mountain, the world is silent. If you see the mountain and forget the Tao, the mountain is noisy." Do you know where this ancient poem comes from? Can you understand its meaning?
This sentence is from Yue Xin Ji, a transcript of Yong Zhengdi’s reading in the Qing Dynasty. It can be understood as follows: If a person forgets the beautiful scenery of the mountain when he sees the Tao, then he has seen through the world of mortals and feels lonely even when he is in the world; If a person forgets the Tao when he sees the mountain scenery, he will feel noisy and impetuous even when he is in a quiet mountain.
For many people, ancient texts that have never appeared in such textbooks are slightly obscure.But today’s AI technology has been able to understand these sentences which are difficult for ordinary people to understand, and can also appreciate the artistic conception and draw the matching "Chinese style" paintings.
Recently, the team of Professor Wen Jirong, executive dean of Gao Xuan College of Artificial Intelligence of Renmin University of China, published the AI model named "Wen Lan", which can understand Chinese language, including ancient Chinese and ancient poetry, and turn it into paintings. This achievement was published in the top scientific journal Nature Communications.
Today, AI painting is becoming more and more popular, and the research results show the world the profound meaning of Chinese language and the charm of "Chinese style" painting.
AI painting has been popular all over the world recently. From technology giants to creative individuals, all kinds of painting AI have been launched. Although the works created by these AI technologies are flawed, the overall effect is amazing, and the paintings created by AI won the championship in the art competition, which made some artists feel dissatisfied.
But the painting AI developed by these westerners is not necessarily suitable for Chinese context. In terms of painting style, most of the works created by painting AI from the west are realistic, while Chinese painting pays more attention to "freehand brushwork". Black and white ink painting is also different from colorful western painting.
In terms of language, the meaning of English words is relatively clear, and words and referents correspond one to one. Therefore, painting AI in English context generally uses "image-text" pairs with strong semantic correlation for training. However, Chinese words are composed of a single Chinese character, and the same Chinese character may represent different meanings in different words. The "image-text" pair with strong semantic relationship is not suitable for training Chinese painting AI.
However, ancient Chinese is more difficult than modern Chinese, and it is even more difficult to develop an AI that can paint according to ancient Chinese, in addition to understanding the artistic conception behind poetry.
In view of the implicit and rich connotations of ancient Chinese and Chinese paintings, Professor Wen Jirong’s team innovatively chose to train AI with weak semantic-related "image-text" pairs, and achieved excellent results.
Specifically, they combined the self-developed multimodal pre-training model "Wenlan" with the latest image generation technology, and pre-trained 650 million Chinese "image-text" pairs with weak semantics, so that they could learn the unique and profound meaning of Chinese and connect Chinese semantics with visual information.
During the training process, they guided "Wen Lan" through iterative generation. First, they randomly initialize a "noise picture", and then after each step of program processing, the generated painting is closer to the input text, and finally the generated painting is close to the input text.
In the end, they successfully developed a painting AI that can understand ancient Chinese and be familiar with China culture, and showed this achievement to the whole world, filling the gap in the field of "Chinese Style" painting AI.
How strong is Wen Lan’s ability to understand ancient prose? Let’s have a look at its works.
The research team first entered four Chinese characters: "Nature", "Surfing", "Campus in Winter" and "faithful wife", and got the following paintings respectively. The images at the bottom are more complicated than those at the top after iteration.
There are some more colorful works, each full of fantastic colors.
They also used "Wen Lan" to draw the life of caterpillars that eventually break cocoons and become butterflies.
For ancient poetry, "Wen Lan" created such works. From left to right, they are "faithful wife", "Being born like a summer flower", "Three or two peach blossoms outside the bamboo" and "mountains cover the white sun, and oceans drain the golden river".
If you think these poems are too simple and the works of "Wen Lan" are not stunning enough, take a look at these:
As mentioned at the beginning of the article, "When you see the Tao and forget the mountain, the world is quiet, and when you see the mountain and forget the Tao, the mountain is noisy." "Wen Lan" created such a work: a tourist in a long robe stood in a mountain with one hand on his chest, gazing into the distance. It must have been the magnificent and profound scene in front of him that triggered his thinking on the philosophy of life.
Compared with the works created by other painting AI for this poem, we can see that "Wen Lan" has a profound understanding of ancient poetry and Chinese painting:
It can be seen that for the three ancient poems, there are no obvious and specific people and characters in the works of "Wen Lan", but they all rely on the scene to present the meaning of the poems, and the scenery also extends to a deeper place, more reflecting the beautiful style and far-reaching artistic conception.
I believe that western AI technology developers will not only marvel at the beauty of "Wen Lan" works, but also admire the profound cultural heritage and top-notch R&D technology of China researchers. After all, it’s not easy for western real painters to create such "freehand brushwork", let alone let AI create after reading the difficult ancient Chinese.
If the painting AI from the west has impacted the creation of modern artists, and may even make them lose their jobs, then today, with the gradual loss of Chinese painting, this kind of painting AI with "Chinese style" will help to stimulate the interest of the younger generation in Chinese painting, make the Chinese painting with a long and profound history shine again and pass it on, and shine brilliantly in the international cultural field.