3D光场重建率先被中国团队推向零售商用！谷歌同款技术，哈佛高材生创办，高通投资认定
3D light field reconstruction has been first pushed to retail commercial use by a Chinese team! The same technology as Google’s, founded by a Harvard graduate, and recognized by Qualcomm’s investment.
模拟人眼，所见即所得
Mimicking the human eye, what you see is what you get.
杨净 发自 凹非寺
By Yang Jing, from Aofei Si.
量子位 报道 | 公众号 QbitAI
Reported by Quantum位 | WeChat public account QbitAI.
最近，大家都在讨论元宇宙。那是否有想过，在元宇宙里应该会如何购物？
Recently, everyone is talking about the metaverse. Have you ever wondered how shopping should work in the metaverse?
是像这样，在手机端就能看到跟线下店一模一样的完整实物？
Would it be like this, where you can see the exact same physical product on your phone as in a brick-and-mortar store?
还是足不出户就可以美美的线下店？
Or would it be a beautiful offline store you can visit without leaving home?
又或者是，看上哪款就试哪款？
Or perhaps, you can try on whichever item you like?
旁友们，是不是以为这就是搞个3D模拟，或者电影特效那种后期制作？
Folks, you might think this is just some 3D simulation or post-production like movie special effects, right?
NoNoNo，其实都不是。
No, no, no – actually neither.
这个用到的是光场技术，模拟人眼，所见即所得。
This uses light field technology, mimicking the human eye – what you see is what you get.
这技术可来头不小。
This technology has quite a distinguished background.
前段时间谷歌I/O大会亮相的裸眼3D通话，就是基于这个原理——
The glasses-free 3D calling unveiled at Google I/O earlier this year is based on the same principle –
将空间里人眼看到的光线进行采集重建，重现真实的三维世界。
capturing and reconstructing the light rays that the human eye would see in a space, to recreate a true three-dimensional world.
如果把它用到我们日常的穿衣住行上，简直“高质量”了有木有！
If applied to our daily clothing, housing, travel, and more – wouldn’t that be "high-quality" indeed!
事实上，已经有公司做到了。
In fact, a company has already made it happen.
而且还是一家中国公司——奥本未来，哈佛团队班底，那几张画面正是他们与几大品牌商的合作。
And it is a Chinese company – Oben Future, with a Harvard team background. Those images shown are from their collaborations with several major brands.
那么具体是如何实现这种逼真感呢？
So how exactly is this sense of realism achieved?
什么是光场技术？
What is light field technology?
提到光场，就要说到我们的视觉感知。
When talking about light fields, we must mention our visual perception.
人类能看到所在的世界，就是通过人眼，来接收现实场景中的光信息，从而进行感知。
Humans see the world around them because our eyes receive light information from real scenes and perceive it.
如果我们能把空间中的光都记录下来，那我们就可以从任意角度、任意路径，去观察，去体验这个空间，就像到现场一样。
If we could record all the light in a space, we could observe and experience that space from any angle and any path, just as if we were actually there.
元宇宙电影《Ready Player One》里，主角进入前人的记忆信息，各种漫游，甚至查看细节来破案，要真的实现这样的场景，就需要用到光场技术！
In the metaverse movie Ready Player One, the protagonist enters the memories of a predecessor, roams around, and even examines details to solve a mystery. To truly realize such a scenario, light field technology is needed!
光场技术，记录了七个维度的信息。
Light field technology records information in seven dimensions.
他们分别是场景中任意点的位置（x, y, z），用来表示它发光方向的水平角度和垂直角度（θ, Φ），发出光线的波长（λ）以及时间（t）。
They are the position (x, y, z) of any point in the scene, the horizontal and vertical angles (θ, Φ) indicating the direction of emitted light, the wavelength (λ) of the emitted light, and time (t).
别小看这七个维度，它就已经能对场景100%光线描述。
Don’t underestimate these seven dimensions – they already enable a 100% description of the scene’s light.
人看到什么，它就捕获到了什么，所以能根据观看需求任意调整视点、焦距和景深。
Whatever the human eye sees, it captures, so viewpoints, focal length, and depth of field can be adjusted arbitrarily according to viewing needs.
实际上光场技术之外，也有其他的方式来记录空间信息。
In fact, besides light field technology, there are other ways to record spatial information.
比如三维建模。相信玩3D游戏的旁友很熟悉了。要实现逼真的3D模型，且不说逼真的实时光照需要巨大的算力，单就光照、材质、几何等要有极高的保真度，就已经极具挑战性了。
For example, 3D modeling – familiar to those who play 3D games. To achieve realistic 3D models, not to mention the enormous computing power required for realistic real-time lighting, just achieving high fidelity in lighting, materials, and geometry is already extremely challenging.
或者使用一些摄影测量或者三维扫描的设备，但更多细节无法捕捉到，离电影里逼真的渲染效果相去甚远。
Or one could use photogrammetry or 3D scanning devices, but many details cannot be captured, and they are far from the realistic rendering effects seen in movies.
所以光场采集无疑是短时间内最快实现商业化的解决方案。
Therefore, light field acquisition is undoubtedly the fastest solution for commercialization in the short term.
奥本未来就是研发的这项技术，有这几方面的的优点。
Oben Future has developed this technology with the following advantages.
首先，采集设备比较成熟。只要一部手机就可以做光场的采集。
First, the acquisition equipment is relatively mature. You can capture light fields with just a smartphone.
以往学界做光场采集，动辄就是几百个相机的阵列。
In the past, academic light field acquisition involved arrays of hundreds of cameras.
要么是由几百个小摄像头集打造的光场相机。
Or light field cameras made up of hundreds of small camera modules.
但奥本未来的技术团队表示，
But Oben Future’s technical team says that
采集过程可以使用任意相机，苹果手机后摄也可以。
the acquisition process can use any camera – even the rear camera of an iPhone.
但为了保证照片质量，一般使用单反相机。
However, to ensure image quality, they generally use a DSLR.
此外，技术人员还透露，通常一件典型的商品只需要采集一两分钟的视频或者一百多张照片。
Additionally, the technicians reveal that typically a single product only needs one to two minutes of video or about a hundred photos for acquisition.
以往光场数据十分庞大，想要在屏幕上完整渲染出来，需要高性能主机才能做到。但如今通过压缩算法，普通网页也能实现逼真效果。
In the past, light field data were so large that rendering them fully on screen required a high-performance computer. But now, with compression algorithms, even ordinary web pages can achieve realistic results.
比如，京东APP上试穿三维运动鞋，就用了奥本未来的光场技术。
For example, trying on 3D sneakers on the JD.com app uses Oben Future’s light field technology.
还有在unity做的三维游戏也用到了光场模型，动起来的那种。
Also, 3D games made in Unity have used light field models – the kind that moves.
正是基于以上这些技术突破，奥本未来让真实空间和物体的整套流程变得更加高效。
Based on these technological breakthroughs, Oben Future has made the entire process of capturing real spaces and objects much more efficient.
目前，奥本未来自研了一款光场捕捉器，最小化人为干预，只需连接普通相机，即可快速自动化完成对常见物体的光场采集。
Currently, Oben Future has developed a self-designed light field capture device that minimizes human intervention. Simply connect an ordinary camera, and it can automatically complete the light field acquisition of common objects.
在面对海量SKU的电商行业，从采集、重建、部署到上线平台整套下来只需要一天就可以完成。
For the e-commerce industry, which deals with massive SKUs, the entire process from acquisition, reconstruction, deployment to going live on platforms can be completed in just one day.
技术人员透露，当中成本只包括两部分：拍摄照片的人力和云端重建计算的算力，相当于传统三维建模的十分之一。
The technicians reveal that the cost includes only two parts: the labor for shooting photos and the computing power for cloud reconstruction – about one-tenth of the cost of traditional 3D modeling.
不过，他们坦言，当前仍然具有诸多挑战，比如，大规模场景重现。
However, they admit that there are still many challenges, such as large-scale scene reconstruction.
跟高奢品牌打交道的奥本未来
Oben Future, working with luxury brands
最后，再来简单介绍一下奥本未来。
Finally, let’s briefly introduce Oben Future.
据官方介绍，它是一家用前沿技术来解决三维内容供给，并致力于将实时三维交互推向更多行业背景的技术公司。
According to the official introduction, it is a tech company that uses cutting-edge technologies to solve 3D content supply and is committed to bringing real-time 3D interaction to more industries.
团队方面，奥本未来由哈佛大学海归团队于2016年创立，创始人兼CEO雷宇博士是一名连续创业者，博士毕业于哈佛大学应用物理及材料科学系。
The team: Oben Future was founded in 2016 by a returnee team from Harvard University. The founder and CEO, Dr. Lei Yu, is a serial entrepreneur with a PhD from Harvard’s Department of Applied Physics and Materials Science.
CTO沈方阳博士毕业于北航，虚拟现实领域博士，曾在微软亚洲研究院、哈佛可视化计算组（Visual Computing Group）从事计算机图形学和视觉方向研究
CTO Dr. Shen Fangyang earned his PhD from Beihang University in virtual reality. He previously worked on computer graphics and vision research at Microsoft Research Asia and Harvard’s Visual Computing Group.
此外，技术团队还来自于哈佛大学图形学实验室、北航虚拟现实国家重点实验室、微软亚洲研究院、清华、北大、北邮等各高校机构人员组成。目前也在积极广纳英才，比如图形学程序员了解一下？
In addition, the technical team includes members from Harvard’s Graphics Lab, Beihang’s State Key Laboratory of Virtual Reality, Microsoft Research Asia, Tsinghua, Peking University, Beijing University of Posts and Telecommunications, and other institutions. They are also actively recruiting talents – for example, graphics programmers, anyone?
融资上，团队们上轮完成了Pre-A轮融资，由高通、京东领投。此前的天使轮还包括华创资本以及多家清华系基金等投资机构。
In terms of financing, the team completed its Pre-A round led by Qualcomm and JD.com. The earlier angel round included investments from China Creation Ventures and several Tsinghua-affiliated funds.
截至目前，奥本未来已经落地服务超过30家品牌客户，都是大家耳熟能详的著名品牌。
To date, Oben Future has served over 30 brand clients, all well-known names.
当然也有一些接地气的平台合作伙伴，比如京东、微信小程序、天猫等主流电商平台。
There are also more down-to-earth platform partners, such as JD.com, WeChat Mini Programs, Tmall, and other major e-commerce platforms.
而之所以跟品牌方、电商平台打交道，源于2019年的洞见。
The reason they work with brands and e-commerce platforms stems from an insight in 2019.
随着用户对电商购物的体验要求越来越高，3D互动导购内容和场景开始大量增长，
As users’ expectations for e-commerce shopping experiences rise, 3D interactive shopping content and scenes have grown significantly.
相比与以往图片、视频等有限视角，3D素材、3D实时交互以及AR试穿试戴素材相关的需求只会越来越高。
Compared with traditional images and videos with limited perspectives, the demand for 3D assets, real-time 3D interaction, and AR try-on/try-out content will only increase.
直至现在，电商随处可见3D内容场景，比如商详页、频道页、店铺页、直播间、搜索页、会场页等等。
Now, 3D content scenes are everywhere in e-commerce – product detail pages, channel pages, store pages, live streams, search pages, event pages, and more.
对于品牌电商来说，一整套低成本、门槛低、轻量级的三维内容方案成为他们的首选。
For brand e-commerce, a low-cost, low-barrier, lightweight 3D content solution is their top choice.
比如，便携式采集技术和低门槛的三维编辑工具可以帮助品牌与内容制作者节省制作的成本；
For instance, portable acquisition technology and low-threshold 3D editing tools help brands and content creators save production costs.
整个编辑存储发布都在云端，有助于多方协同；整个方案可以支持多平台发布的属性。
The entire editing, storage, and publishing are in the cloud, facilitating multi-party collaboration; the solution also supports multi-platform distribution.
这么说起来，奥本未来所做的更像是一套底层基础设施，推动整个购物体验数字化、沉浸化、智能化。
In this sense, Oben Future is more like building a foundational infrastructure to drive the digitalization, immersion, and intelligence of shopping experiences.
如今元宇宙概念的爆发，似乎也印证了奥本未来之前的选择。
The recent explosion of the metaverse concept seems to confirm Oben Future’s earlier choices.
至于实现真正的元宇宙，需要更多像奥本未来的公司来共同打造技术底座。
As for realizing the true metaverse, more companies like Oben Future are needed to jointly build the technological foundation.
