The 3D Avatar Store
Landing page to create an avatar teaser
An ad we placed on other websites
In-browser 3D animation controls teaser
Exported 3D Avatars integration into other software
Celebrity 3D Avatar demos
Dedicated high compute 3D Reconstruction servers
Located in former Enron Datacenter, we added Federal Reserve Bank security
Also built private server clusters for requesting clients
Way back around the summer of 1976 I was age 12, and being a geeky kid I was at a university lecture describing the brand new field of 3D computer graphics. Watching early 3D stick figures animating at that lecture, I imagined a day when video games could have realistic 3D characters. This being the days that a home video game was Pong. I imagined those future video games could have your own face on the character. It seemed obvious to me to put the player's likeness on the game characters. I decided to pursue 3D graphics and animation as a career soon there after.
Over time I became a professional software developer, with specializations in 3D graphics, video/data streaming, 3D video games, and animation media production pipelines. After nearly 20 years developing software, I entered the Feature Film VFX Industry as a technical digital artist while simultaneously attending an MBA program. My ambition was to acquire an MBA and launch a startup. Expecting the MBA to be harder, I found myself competing for head of my class. I also found I had an aptitude for VFX, moving to a technical artist supervisor role, then a production financial analyst, and finally a production fire fighter role where solutions requiring deep knowledge of 3D programming, internal studio tools and staff's varied capabilities are used to solve productions facing deadlines.
It was doing this production fire fighter work that I became somewhat of a specialist in stunt double actor replacement. Within feature film VFX, stunt double actor replacement is a very common type of job. Consider how much money a film production loses if one of their stars is injured and production is delayed; for this and many more financial reasons actors rarely do their own stunts. During the final days working on Disney's Ice Princess that I realized I knew all the pieces and parts necessary to realize my 12-year-old-self's idea of video game characters with the appearance of their game players. Thinking deeper, I realized a larger market is TV commercials, music videos and film trailers where actor replacement is used to insert ordinary people into the everyday advertisements they see while viewing their daily media.
At this time, about 2003, cable TV being digital was yet new and had for the first time the ability to individually target households with demographic targeted advertising. Like the cocktail party effect where one can hear their name spoken in a noisy space, I envisioned video advertising where you appear in the ad itself, along side a celebrity spokesperson declaring how smart you are for using some product. Such Personalized Advertising would stand out as a video version of the cocktail party effect. I imagined desirable music videos and major release feature film trailers fan consumers would be willing to pay a nominal fee, such as $1, to own a version of their favorite media with them embedded in the action as if the consumer is a star along with the celebrities in the real media.
My MBA program had an option to write a graduate thesis. I chose to write a thesis with the topic being a business plan and strategies to create a new branch of the Advertising Industry featuring Personalized Advertising with the consumer inserted into the advertising media. During research for my thesis I attracted interest from a few VFX industry friends. After graduating in 2004, 2nd in class with a double major in Finance and International Business, plus my Personalized Advertising thesis, three VFX industry friends and myself formed a company we named Flixor. Our goal being an advertising agency specializing in Personalized Advertising: video production with the viewing consumer actor-replaced into the media.
It took time before public demonstrations and investor pitching began. First priorities were implementation of an actor replacement production pipeline, as well as securing our intellectual property with patent protection. The founding team included Liz Ralston, VFX producer of the Academy Award winning film "Babe: the talking pig", Todd Shifflett, a VFX Supervisor of multiple 3D character driven films, and Miles Lightwood, one of the architects of Disney Online. Together we initiated an international patent strategy, began researching potential technology partners, and began developing our VFX pipeline.
We landed Fish & Richardson as our patent attorneys, receiving the same patent attorney as Apple and requiring a special waiver from Apple declaring they were not working on any variant of automated 3D actor creation or actor replacement in media technology. Our patent strategy was a global patent covering the national and international patent regions with the largest Internet consumer penetration. Due to foreign patent requirements to be expressed in the language of their region, we hired two independent native foreign language speakers for each foreign language. Each of these individuals were also digital media software developers. I wrote the main text of the patent, which was modified my our Fish & Richardson attorney for accurate legal language, then translated into each foreign language by the first foreign language developer, that gets modified for accuracy by our Fish & Richardson attorney in that patent jurisdiction, and finally that gets translated back into English by the 2nd foreign language developer. The translated-back-to-English version is reviewed by myself to insure the essence of our technology and innovations were not lost or mistranslated. The entire process of creating and filing seven patents, six carefully crafted in highly technical non-English languages, required two years. However, our detailed process paid off, as our patents began to be awarded across the globe with minimal and minor modifications.
Meanwhile, the VFX pipeline received an initial implementation with a custom 3D Render dedicated and only capable of our actor replacement VFX process. Due to this specific purpose, the not at all generalized 3D renderer was capable of optimizations not available to general purpose 3D Renderers. Our optimized Flixor Renderer was capable of rendering standard browser advert sized video clips at 12 times real time, which equates to the same actor replaced video being rendered with 12 different people in 12 different versions of the clip simultaneously. This produced a scalability for the service such that large ad campaigns with personalized versions numbering in the tens of millions was technically and economically possible.
We still lacked a method of automatically creating a 3D version of a specific person. In the VFX industry actors are sent to dedicated 3D scanning facilities. Research led to a facial recognition plug-in developer offering an SDK for facial recognition developers for performing 3D reconstruction of a face from a single image. In facial recognition this technology is used to virtually turn a subject's face towards the camera. I saw it as the basis of automatically creating 3D models of real people without expensive 3D scanning. After negotiating, we formed a partnership with CyberExtruder, where they created a dedicated version of their neural nets trained to generate a VFX quality 3D geometry we provided when performing a 3D reconstruction. In exchange they receives a percentage of our 3D Avatar creation and exporting revenues.
A simple buy or lease calculation indicated the high compute requirements of our startup could run $96K per month if implemented at Amazon AWS, or we could assemble our own data center for 2/3rds of that and pay $600 a month to co-locate at the former Enron data center. Choosing to buy our own servers, I wrote a dedicated 3D Reconstruction Server capable of 25,000 unique 3D Avatars per hour. Using the Reconstruction Server as basis, I created a bare metal server cluster capable of 125K unique avatars per hour per cluster with hardware fault tolerance, multi-tiered data resilience, and integrated into the (then in wide use) Drupal eCommerce platform modified for custom created on-demand products.
We begin pitching to venture capitalists, film and music publishing studios. Initial meetings with venture capital are met with dismay and disbelief: being early 2008, only one VC we pitch believes automated actor replacement is possible at all, much less possible at Internet scale. The reception at film and music studios is better, they sign letters of intent to be customers of the service once available, but decline to invest, saying we should go to silicon valley VCs for investment By this time, the 2008 global financial crisis has begun, and the one VC who showed interest declines continued discussions, despite our numerous letters of intent. My founding partners leave the startup, and I choose to pivot to a personalized 3D game character creation service.
After a year and a half of rewriting the infrastructure, I relaunch as the 3D Avatar Store mid 2010. During this period I earn additional income by writing business plans, financial statements, and web sites for other startups. I continued pitching, but to Angel Investment Groups and as a custom 3D game character creation service. Three different times a team of Angel Investors is formed for the company - only to have at one point one of the investors suddenly realize what our technology could do if applied to pornography. Soon the entire Angel Team is insisting the company create pornographic media. Granted, at the time the conventional wisdom was that the porn industry introduces new technologies, but in this respect I believed our technology would create a lawsuit engine and be detrimental to the public. I refused, and eventually disbanded that Angel Group. Three times this happened. I compiled a good sized list of positive and profitable applications not using porn, but the Angels insisted and I continued to refuse.
All the while, I continued to optimize and add features to the 3D Avatar Store's capabilities. The slides at the top of this post show how far I pushed this project.
Maintaining global patents with minimal income from the 3D Avatar Store is the issue that finally ended my efforts. We managed a few small game studio clients, we had a medium sized number of general public users, but never managed enough revenues to break even with our ongoing patent expenses. During the years of effort, CyberExtruder creates more custom versions of their software for us, and each time tries to hire me away from my own startup. Finally, going personally bankrupt I accept CyberExtruder's offer and sell the patents to escape complete bankruptcy. It is fall of 2013.