A Retrospective on TextSpark.ai

A Retrospective on TextSpark.ai

Let's talk about TextSpark and startups today.

Lately I've mentioned having to make some hard decisions about how to invest my time, and I've been agonizing over how to channel my energy lately with just Too Many Things on my plate. 

For the last six months (or year if you count the early experiments playing with GPT-2) I've been working A LOT on getting this AI creativity tool rolling, and you've probably heard me talk about it.

Full AI writing is coming soon. I'm super interested in it. 

And this summer when I was invited into the GPT-3 beta and saw how impressive it was at context retention, I decided I wanted to build a tool around it since I was already using something similar in writing experiments. 

Other than my publishing business (which was a whole different animal) this was the first business I've ever actually taken from concept to commercial venture, and it's been a really interesting and educational experience. Wild ride so far. 

Originally, TextSpark was intended to use GPT-3 under the hood and we were working closely with our OpenAI contacts throughout July and August to prep for a beta period in September. I was worried about competition in the space and wanted to hit the market running. 

Alpha feedback was fantastic from users and I was excited about the applications for fiction writers that would no longer need to have a lot of technical know-how to leverage AI tools in their writing process (eliminating a technical legibility gap that existed). 

My partner and I were pouring a TON of time into the project (I was pulling 60-100 hour weeks over that period) to get it market ready and lining things up to make sure all of our biz stuff was ready to go. 

The policy at the time for OpenAI was you had to actually build the application and have it production ready to submit to their review process, but I wasn't too concerned about the investment because they'd had my design docs and I was chatting with them throughout. 

Unfortunately, in September, the conversations (and review process) halted while they revamped their use policies. I spent a few weeks begging for an indication of what that would mean for our intended use case so I could pivot our plan if it was unsupported. 

They couldn't give me an indication either way--either yes or no, so clearly there was a lot of internal debate around the safety of unfettered text creation (which I get). Frustrating from a business perspective, but so it goes. 

I was told there was a "range of possible outcomes," so we pressed forward and hoped for the best, continuing to get ourselves ready to move from alpha to beta once they opened reviews again and gave us a decision on the app we'd submitted. 

Unfortunately, when the air cleared on their decisions around use applications, it turned out that unconstrained text generation was a clearly banned use case. This made our design totally unworkable for the flagship use case we'd built around. 

The complete block on the use case, regardless of what safety measures we were willing to put in place, came as a bit of a shock to us. It was a difficult message to hear when we'd been pushing so hard. I almost lost my business partner over it. 

But OA was playing a much larger game than we were and it was their tech backing us, so at least it was a fair decision (and I can respect that). Disappointing, but fair.

We decided to regroup and look at other options. 

When I saw @gwern's poetry results with GPT-2, I decided to take another look at the largest model as a possible replacement and see if we could train it as the primary creativity tool while still using GPT-3 for more constrained generation cases. 

I worked with a helpful and capable guy who I won't tag here because I don't know if he'd want me to in order to tune GPT-2 toward fiction writing, and after a bunch of testing we got the model and the params to a place where it was producing REALLY impressive results. 

They weren't as good as GPT-3 vanilla, but I'm proud that we managed to land probably 70-80% as good of text gen results in terms of context and content as a dramatically larger, better, and more expensive model. 

In the meantime, I continued working with OpenAI to try to get clarity on whether we could launch "idea generators" which used much more constrained prompts to generate backmatter, story ideas, titles, characters, etc. 

We'd been told that they'd review different aspects of the design separately but had only gotten a blanket "no" on the app, so I pressed for more details and a full review.

In OpenAI's defense, I'm sure they were ridiculously slammed with review requests (and continue to be) .

But when the review came back on the generators, I still didn't have clear guidance on whether we could move forward or not.

We had pre-built all of the technology because they asked us to submit a working app for the review process rather than designs and descriptions. 

The feedback we got didn't make any sense. I had handed them a functional app with full descriptions of how it worked for either a greenlight or not, and they came back with responses that said "likely" or "not likely" to be approved with some really inconsistent reasoning. 

They said more information was required for approval, but didn't specify what information or ask for follow-up details and I honestly didn't know what more to give them.

I was stressed, tired, and upset at this point, trying to keep my cool. 

I was having conversations with their product lead, with their PM lead, with their CTO, with our account rep, just trying to get this sorted. It felt very frustrating and disorganized for a business partnership, even if we were small potatoes. 

I sent a long email respectfully expressing my frustration and confusion with the adhoc process and tried to address each of their concerns directly, providing more information as they'd asked and trying to get clarity on the issues.

I asked if there was any path forward. 

No one ever replied to that email despite all of my contacts at OpenAI being cc'd, so it was a clear enough message for me.

I was still pretty frustrated, but I dropped it and decided to focus on GPT-2 and what I could control instead. 

After all, they had a full plate and we were just one use case (and one they clearly didn't really want to support), so it didn't seem fruitful to continue pursuing that partnership.

I was also aware of other solutions coming to market and had confidence in alternatives. 

We made a full pivot away from our original plans and focused on self-hosted solutions for the GPT-2 model we'd trained which was still producing really great results for both our textgen and our generator use cases. 

But it quickly became clear that this was going to require a lot more time, investment, and work on both the development and management side than simply building a front-end around a third party API (which had always been the plan). 

My partner and I had already discussed that we'd make a huge up front investment of time and effort to get it off the ground and chill and see how the response was while we decided what next steps were.

Now we were looking at an ongoing major investment of time and effort. 

We had a frank conversation about what we could both put into the project and my partner decided it was too much of a risk to divert that much time to without a clearer signal of product-market fit, but I wanted to press forward and try anyway. 

He graciously agreed to get the tech to a place where he could hand it off to me and I was planning to press forward solo from there. This was daunting as he was my technical partner and I was tech-savvy but had never done anything on this scale before. 

It was overwhelming, to be honest. Suddenly I was trying to run a startup solo while teaching myself enough to both continue developing the project and also run the company. 100 hour weeks became a norm for a while. 

I realized I needed support since it was too much to learn too quickly in a vacuum and I phoned an old developer friend who agreed to work with me on it at a deferred hourly rate, and he was absolutely awesome.

He helped me improve it and mentored me on coding. 

My coding skills have improved by 1000% EASILY thanks to working on this project. I now know how to do things solo I wouldn't have known how to begin six months ago. Such a cool experience. I can't recommend this type of exercise enough. 

So we got it to a place where it was commercially viable, and now I had a new problem: We had a viable product to pitch to either B2B or consumers.

Like a dummy, I decided to focus on both in spite of all the advice not to do that. 

This presented both product AND pricing issues for us, because the value of accelerating a technical writer for a firm is very different from the value offered for a creative tool to help indie authors brainstorm ideas for their stories.

Different tools, different pricing. 

I couldn't both pitch an appropriate valued B2B accelerator and a very similar product being offered at a much lower price point to B2C consumers. What I should have done was focus exclusively on B2C (as was my original intent). But there was a lot of business interest. 

That was a lucrative distraction. I tried to split my focus by hiring a sales VP to focus on customer qualification for B2B while I was going to focus on developing the B2C application.

But he started pulling in partners that were flashing dollar signs at me. 

But my sales VP, while friendly and smart, just didn't have the immediate expertise to demo and pitch the product sufficiently yet. We were running at full speed and learning on the go.

So I had to lead a lot of sales calls too. 

The sales calls were another very big distraction, and we were taking a lot of them. My weeks were booked solid with pitch meetings. My nights were coding until 2am, answering emails, and managing administrative stuff. 

We were getting very, VERY strong interest from the firms we were pitching who were delighted with the tool on first contact when we demo'd it, but we were struggling to actually land the deals.

I didn't think it was price OR quality of the generated text. 

What I suspected, and what I still suspect, is that we were running against two major factors: First, writing collaboratively with AI is it's own skill you have to build (and I've written three articles on this so I know), and second, you still have to know what to write. 

Even GPT-3 won't write FOR you. It requires both editorial skill and writing skill to know how to shape the responses in a way that actually accelerates you. Lacking these skills makes it harder than just writing to use the tool. 

What came out of our conversations was that that was what our business partners really wanted--they wanted something that could automate the process of content creation. Not accelerate an already talented writer (who would have to learn new skills). 

This was frustrating for me, but it wasn't their fault. I'd built a cool gee-whiz tool that I knew COULD accelerate them, and they agreed it was very cool, but we couldn't get buy-in from their operators and thus couldn't demonstrate value outside of when I was demoing it. 

This was really on me as the CEO and designer for not making it user-friendly enough. I was solving a problem, but not a pressing problem. Their hair wasn't on fire over the cost of producing content. So it was an uphill battle to close the deals. 

And when we launched the consumer product, my price point was too high for the consumer market. I knew this. But I still wanted to get something out there with the original vision in mind--I just didn't want it to undercut our business deals. 

My solution for this was to make business-specific tools tied to business accounts that differed from B2C author tools. More development work. More design work.

I built a VERY cool social media post generator that wowed everyone we showed it to. 

Still very proud of that. It's a neat tool.

But once again, it's not a problem that lights anyone's hair on fire. It's COOL to generate 100s of possible tweets for your brand in seconds.

But it doesn't solve a pressing enough problem to open checkbooks for large sums. 

Also, I think it muddled our brand messaging. It was difficult to articulate the value package in a way that resonated with our potential clients enough. They'd come in, play with it, agree it was neat, but then just go back to their normal ops process because it was easier. 

Meanwhile I was trying to sell an overpriced B2C author tool that frankly scared a lot of authors and made others bristle at the suggestion that their work might be improved by working in conjunction with an AI. 

I've had some very ...interesting... calls with authors over the last six months. Some people think it's very cool. Some people want me to light the app on fire and sink it in the ocean (as though this will prevent AI from entering writing). 

On the business writing side, owners love the prospect of having their operations accelerated, but we get massive resistance from writers who feel it's an insult and an encroachment on their jobs and expertise. Difficult to sell both the CEO and the operator. 

In spite of tons of enthusiasm and positive feedback from early meetings, after about two months we still hadn't closed any initial B2B customers (at any price point) and I was still leading our B2B sales calls on top of everything else I was doing. 

My VP was full of enthusiasm and big numbers, but he wasn't really spinning up new client contacts or closing any more than I was, and I was still clocking 100 hour weeks for the third or fourth month in a row (still also running my other business and other projects). 

I knew we had a product-market issue on the B2B side but wasn't sure how to improve it with the feedback we were getting from potential clients. They weren't able to articulate any particular issue.

Again, this tells me we weren't selling the right solution. 

I was getting very tired of taking sales calls that weren't going anywhere (because I can do it, but I hate selling--that was why I had my VP). He was getting frustrated too. He was trying his best and putting a lot of time in. 

But as I started considering a hard pivot back to the original B2C vision since I didn't really want to go searching for a problem in the B2B space, my VP started talking about managing a promotional strategy for B2C which was not what he'd been hired for. 

We had another hard conversation and it became clear he was losing faith in the B2B space too, and I explained that I couldn't support a sales role in the B2C space given margins there. We agreed to part ways and I pivoted back to B2C focus. 

That had cost me months of focus and effort and I was already exhausted. I also still didn't have a B2C payment system in place, because I'd been focused on B2B which would have been manageable via direct billing. 

This was when you all got to watch me struggle with Stripe for the better part of six weeks, once again stretching me SO much further than I'd ever been stretched before as I worked to integrate Stripe natively into our platform. 

Fantastic learning experience, but still burning the candle on both ends. I was still doing admin, still managing contractors and volunteers, still managing my other business, and still taking sales calls in the hope that maybe one would come through. 

Throughout this period I got NUMEROUS offers to invest in TextSpark. Friends, old colleagues, strangers, institutional investors. I turned everyone down because I didn't want to take money until I knew we had a product that would resonate with our audience. 

I know myself, and it just would have been an anxiety weight around my neck. I didn't need additional funding to develop it. What I needed was more confidence we'd built the correct solution for our audience.

I didn't wanna take money I wasn't sure I could pay back. 

If I had had more evidence of fit on the B2B side, taking funding would have been a no brainer. But it also would have locked me into a path where I felt responsible to my investors, and I didn't want the extra pressure unless it was clear we were riding a rocket. 

Integrating Stripe burned me out finally. I got it done, but I was just tapped out. The B2B stuff had been a huge disappointment and I didn't really know how to improve that side without a lot more investment. 

I felt bad that my sales VP hadn't made any money. I felt guilty that my developer hadn't made any money. He kept reassuring me that he was just doing it for fun and to help a friend, but it's important to me to pay my debts and make sure people are taken care of. 

And I felt embarrassed that I had put so much time and effort into building the wrong thing (apparently).

I decided to take a step back and reassess the whole situation once Stripe was done. I needed a breather. 

The consumer launch went really well, all things considered. I was still working on features a little bit in spite of being exhausted, planning for next steps, and we were picking up a few customers even. 

I got a lot of good feedback on the product. My judgment of the quality seemed dead-on. For its intended purpose as a creative tool, it worked great. Lots and lots of people logged in, played with it, came back, sent notes. 

But too many people came in, tried it, and then didn't return for me to have confidence that we had found product market fit. We had customers, but a VERY low conversion rate that was out of whack with what I had expected.

I spent a few weeks trying to figure out what was up 

Upon reflection and observing usage patterns, I have a few theories about what I was looking at.

TextSpark (and GPT-X in general), as I've always maintained, works best as a creativity and acceleration tool in the hands of a skilled writer. 

But skilled writers actually don't really experience a TON of writer's block. It's more about staying in the flow and battling fatigue than it is not knowing what the next line should be.

So for working authors (like me), especially with a clear outline, not that useful. 

And newer writers, who do struggle with those kinds of creative blocks more, face two challenges: 1) not enough revenue from book sales to justify the cost of a tool that accelerates them, and 2) not enough skill to guide the tool effectively in its current state. 

Finally there's a class of aspirational writers who don't actually write much, and my hope is that TextSpark might have jumpstarted them into action and finally given them a push to start working on the novel they've always dreamed of... but I think I was too optimistic. 

People who SAY they want to write a book but don't actually do it (in spite of the many available tools) are not just waiting for the right conditions... they don't actually want it badly enough to put time in. So a tool isn't helpful, because it still requires motivation. 

Once again, the most valuable tool there would probably be something that translated thin ideas into a more fully-fleshed out piece of prose (which was also what our business partners wanted). But TextSpark doesn't do that. 

I had a long conversation over PM with Gwern once about how that's actually gonna be the killer AI writing app... current tech just isn't there yet.

So even with the price adjustment I pushed out to our tiers this week, I think we're going to struggle resonating. 

There are possible ways around this--I follow people here on Twitter and on Discord working on some very cool methods for AI story generation that's more promising than the rolling prompts we use for TextSpark for that use case. 

I also advanced my own model of how you might use something like GPT's classification abilities to self-police auto generated work. I still think that model could produce some really interesting results. I even got another offer to fund experimentation there... 

But the reality is that I have a million things I want to build, study, and do, and I didn't want to charge off in a totally new direction without taking stock of where I am and my goals, so I turned down that offer too. 

So where we're at right now with TextSpark is I do have customers, but I think there's quite a bit of work still to bring the platform to a place where the AI tooling actually offers a product market fit for customers. This would be true even if we had GPT-3 backing it. 

And we're still burning more cash on the server fees and administrative stuff than we're pulling in by a fair margin. As startups go, I have a LONG runway. It's a very low burn rate. More like an ember. But it's an ember that burns every time I remember it. 

And the administrative stuff still eats a ton of my time.

There are number of pivots I could make with it--I've considered a Chrome addon rather than trying to maintain a browser based writing tool on our own. I've considered white labeling it and selling it B2B again. 

But if I take another step back, the focus on the business is actually starting to really frustrate me. I'm still basically working two jobs when I don't have to be.

What I'd really like to be building is my videogame and working on my magic research. 

I don't actually WANT to hustle in the B2B space for the cash when I don't need it. Again, if I thought we had the right product, I'd ride the lightning.

But if I'm hunting for a problem to solve and tweaking and building on this, I think there are better uses of my time. 

Someone else will build interesting solutions in this space if I don't do it--I don't think anyone else is likely to build the types of products I am if I focus on my other spaces any time soon.

This might just be burnout, but I think I need to lay this one down to rest. 

Because honestly, I'm MORE passionate about the other projects than I am about TextSpark. I am passionate about AI writing, but it's a matter of degrees and how much effort and focus I have to spread around.

Pivoting would be a financially-motivated decision. 

But if I wanted to work super hard on something for purely financial reasons that was taking me away from my other passions, I'd just go back to FAANG-land and draw a huge check for my stress and effort.

I already built what I wanted to build here. 

So I think I'm gonna draw back and lay it to rest to focus my efforts elsewhere. It was SUCH a cool experience and I learned so much and improved my skills at coding, management, negotiation, and sales--we brought a product to market and I'm proud of it. 

But for now I'm gonna scale it down, give everyone the chance to retrieve their writing, and move on to my next big venture. IDK really know what else to do with it. I might open source it, might hang onto the code in case I feel like playing with it later. 

I just can't keep trying to do 3+ massive things at once and something has to give. I still think this space is FASCINATING and will continue to keep an eye on it.

But the whole point of springing myself from corporate was to NOT do stuff like I'm doing now. 

It's been a distraction from the more fun and playful things I'd like to be focused on with my free time.

I'm sure I have several more companies in me--this is far from my last attempt. This just isn't the one for me. 😁

So, so grateful to everyone who's helped. 

And also really glad that I didn't take institutional funding to leave myself this door to pull back. If you're interested in TextSpark or you're curious about using the model or tool for something, feel free to reach out. I'm always happy to chat. 

In the meantime, I'm going to take a much-needed break from worrying about it.

For more on AI writing, I'd encourage you all to check out Sudowrite.com which @superamit and @jamesjyu are working on, and also pay attention to Eleuther AI.

They're doing some VERY cool things not dissimilar to what TextSpark was intended to do and I have no doubt that we'll see some really interesting alternatives in the near future. I've spoken with all of them in the past and have a ton of respect for them. 

As for me, I'm gonna wander off into the desert and get back to studying magic and building videogames which makes me MUCH happier as a general use of my time and effort.

I'll try to hustle up some other company next time I get excited about something. 

As I look back on all of this, there's one major mistake I think I made, and it's very clear to me now:

Our only employee was a hot girl, and we hadn't yet found product market fit.

C'est la vie.