Is it appropriate to use a SaaS like Firebase to handle the data and backend of potentially scalable apps with large amounts of data?

Nothing about Firebase makes it an inherently poor choice for storing large amounts of data. If you structure your data right and access it efficiently, you should have no issue scaling to huge amounts of data.

So, if your concern is limited to “lots of data” no problem…

Keep some other things in mind though.

  1. With Firebase, you just get and set data. There is no server-side processing of the data. You write no server-side “code” other than a few security rules and the like. However, this can be largely overcome by writing what I call a “reactor” server which connects to your Firebase server and r

Nothing about Firebase makes it an inherently poor choice for storing large amounts of data. If you structure your data right and access it efficiently, you should have no issue scaling to huge amounts of data.

So, if your concern is limited to “lots of data” no problem…

Keep some other things in mind though.

  1. With Firebase, you just get and set data. There is no server-side processing of the data. You write no server-side “code” other than a few security rules and the like. However, this can be largely overcome by writing what I call a “reactor” server which connects to your Firebase server and reacts to changes in the data on an as-needed basis.
  2. Firebase is not super-expensive, especially at smaller scales, but there are cheaper options available. You need to weigh the pros and cons of a super-easy, real-time database and BEaaS (Back End as a Service) like Firebase keeping in mind that paying more for your server and hosting does NOT always mean your expenses go up. It may well save you money in the long run if you don’t need to pay as many software engineers to reinvent the wheel, but it could also end up being a large expense on its own and hiring fewer devs won’t always offset that.
  3. You can easily get locked into something like Firebase. There’s always a way out but the further you go down that rabbit hole the less fun it will be to migrate away from Firebase to your own custom server and database architecture. Facebook got so locked into PHP that they completely re-invented the architecture and spent lots of money developing new front-end tools and frameworks to handle their needs rather than migrating to a more robust and contemporary platform. Firebase is an excellent option to get your idea off the ground but keep your escape plan in mind and decide sooner rather than later whether you want to migrate away. You can also do your application development in a way that will make migration easier if/when you decide to, but that will partially detract from the silky-smooth dev flow a fully Firebase-centric architecture allows for.

So yeah, it’s appropriate to handle lots of data going in and coming out, but if you need to do lots of processing of that data, you’ll have to add your own back end reactor anyway. It might also not be the cheapest option long-term, but it’s definitely one of the cheapest options short-term.

I am not really sure what you mean by “large amouts of data”, but I’ll go ahead to answer to the best of my knowledge.

If you’re speaking about storing large amounts of data, Firebase is a decent option to go for, if you’re building a small to medium scale application. When you’re scaling up to a much larger one, one of the major issues to consider will be if you can afford to pay for it.

If you’re speaking about processing large amounts of data, Firebase is not ideal if your application requires continuous processing on server side (especially when data has to be analysed before there is some d

I am not really sure what you mean by “large amouts of data”, but I’ll go ahead to answer to the best of my knowledge.

If you’re speaking about storing large amounts of data, Firebase is a decent option to go for, if you’re building a small to medium scale application. When you’re scaling up to a much larger one, one of the major issues to consider will be if you can afford to pay for it.

If you’re speaking about processing large amounts of data, Firebase is not ideal if your application requires continuous processing on server side (especially when data has to be analysed before there is some display to your users real-time based on the result). This might greatly reduce the performance of your application.

Earlier it was very doubtful with firebase as it’s not feasible solution for large scale apps. But now they have launched a new add-on to the product which is the Cloud Firestore, it’s a complete NoSQL database.

With help of cloud firestore you can build a complex and large scale application.

Useful links:

If you really had the requirement to store Petabytes, you would already know how to store it. So, to answer your question. Firebase is Google technology. Google has capabilities to store an index of the complete web somewhere on their servers. Of course they can handle your “eventually” petabyte data. The question is if you are able to afford it.

The only problem you will have with investors the moment you tell them you are using Firebase is that they will take a good hard look at them, and decide to invest in them instead. Because selling shovels is more profitable than prospecting for gold.

If you are using Firebase you should at the very least be cognizant that you need to monetize your fabulous mouse trap. Tools provided, and I assume you have monetization to go along with all the analytical tools that Firebase provides to delight your potential investors/acquisitors.

Doing the monetization bit is hard, as in building the tools are h

The only problem you will have with investors the moment you tell them you are using Firebase is that they will take a good hard look at them, and decide to invest in them instead. Because selling shovels is more profitable than prospecting for gold.

If you are using Firebase you should at the very least be cognizant that you need to monetize your fabulous mouse trap. Tools provided, and I assume you have monetization to go along with all the analytical tools that Firebase provides to delight your potential investors/acquisitors.

Doing the monetization bit is hard, as in building the tools are hard.

Doing the analytics in house is harder still. It requires real skill. You cannot improvise that kind of skill. And it could take you tears (typo for years) to personally develop those tools, and learn how to use them.

And the bottom line is that you should only be concerned about your value proposition and its monetization (potential or not).

Because if somebody wants to buy an app instead of reverse engineering it, the fundamental value proposition in its market vertical is the only thing they care about. That and time to market. Which means they will not mess with hiring the team and trying to execute on it themselves.

Investors (or corporate purchasers) don't invest on the basis of return. They invest/purchase on a risk adjusted basis.

Lastly, the only thing of value on the balance sheet is the trade mark equity. Don't let anybody tell you otherwise. Among the accounting challenged that is called the brand.


The opposite argument is that nobody is going to pay stupid money for something they can reverse engineer. Running proprietary scripts all the way through the engineering stack, makes entry and duplication much much harder.

You need to accept that somebody will legitimately make that argument. I aint buying, but I am absolutely certain that some dim-witted corporate drone has made the argument to his bosses. Which Miss CEO promptly dismissed with prejudice, on a risk adjusted basis.


Your risk adjusted basis is:

You have to recruit six people with all the necessary technical skills.

You have to recruit another six people to market and sell the mouse trap.

And you have to execute on the basic technical proposition without having verified if your business proposition is sound.

Then you have to execute on your marketing and business plan on the basis of untested technology at volume.

You will have to convince all your peeps to work for peanuts and options to purchase stock at today’s price, for the next eighteen months… GOOD LUCK WITH THAT.

And of course money only grows on the top of the tallest trees, and you look like an ant eater not a giraffe. So God Speed on finding the cash to run your engines as you go up and down down the runway trying to find lift…

So on a risk adjusted basis, 3 months and one quarter the personnel is a far better proposition than 18 months with a large burn rate and a massive execution risk. Because shit happens, often. And when you pay in peanuts you get monkeys: “Sure I can do that…”


So I poured twenty grand into restoring this eighties car I really loved. !@#king Italian sports cars… I might as well develop a coke habit with the hood ornament to go along with that…

Where were we? Ah yes. So I put out five for the car. And twenty for the restoration. Then I find another beauty I want to put my greedy sweaty palms on. And so I list the bad-girl on the net’ and the best I got is fifteen.

Three years later the car is now worth thirty.

Do the math.

The same applies to business propositions. If you invest $2m and two years of tears and blood, that does not mean you have a 2m proposition. Because investors don't invest on opportunity cost. They invest on a risk adjusted basis.

And if a corporate wants to buy your gizmo, lock stock and barrel, they will do so on the basis of the return, or more likely on the basis of the value of your contribution to the perceived value of the corporate stock. Because they are payed in options, and unless something bumps up the value, they cannot profit from the transaction themselves.

So why would you spend money you have no idea you can ever get back? That is the basic proposition for using a tool integrator (and if I was the bastard running Firebase I would put a contract out on me immediately for calling them an integrator…). Speed of execution is everything because building your own tools takes time. And if nobody wants your mouse trap, why spend the money building the tooling to mass produce it?

Speed of execution is EVERYTHING.


Your question presupposes weakness. And the weak get crushed in the market for ideas.

Because what you are saying is that you are done with your project, you have lost interest, and have no intention of doing the dirty work of marketing it, selling it, getting customer feedback and iterating on it. Your are saying you are done.

“I have built it! And so can somebody please sign over a check for a couple of gazillion dollars, because I got better things to do with my time…

And this is the bottom line. You cannot sell an app. Nobody is going to buy it.

If the app has true value to users and one of your competitors decides on a risk adjusted basis that its cheaper for him/her to buy you out, they will come knocking on your door and take you out of your misery.

So no, you cannot sell an app. But you can certainly buy one. Lock stock and barrel and hopefully with the chief nerd in golden handcuffs for six months.

But first things first, build value your users/customers love and love to refer. Because unless you have that, you have nothing of value, regardless of how it was built.

Thanks for the ask, and good luck.

Start with a fast language on the server, minimize the work the server needs to do, and design an easily scaled infrastructure.

If scaling is a major concern, but you start your code in PHP or Ruby, I'm sorry but you're doing it wrong. And yes, a lot of people preach what I consider to be the wrong answer here. I also think that static typing is crucial to solid code. [1] Part of this comes from trying things the other way. I've been developing software for 30 years. I've tried many approaches, and have run into a lot of problems. This is my current wisdom on the topic, in short.

Yes, you can scal

Footnotes

Start with a fast language on the server, minimize the work the server needs to do, and design an easily scaled infrastructure.

If scaling is a major concern, but you start your code in PHP or Ruby, I'm sorry but you're doing it wrong. And yes, a lot of people preach what I consider to be the wrong answer here. I also think that static typing is crucial to solid code. [1] Part of this comes from trying things the other way. I've been developing software for 30 years. I've tried many approaches, and have run into a lot of problems. This is my current wisdom on the topic, in short.

Yes, you can scale slower languages horizontally, but that can be hard to do well, and it tends to give some users a poor experience (bad latency, maybe even errors, while more capacity spins up). When each server can handle 40x as many clients or more, scaling can happen much less often, and if you do implement automatic scaling, there's a lot more buffer between one server being half full and it being saturated.

Look at Quora. They've been fighting a battle with scaling for months now. I periodically can't save a comment or answer because their servers aren't responding. I hear their back end is Python? It clearly has recurring issues. Some of that could be the database layer, which also needs to be scalable.

For the language handling your backend logic, to hit a million “users,” Go is probably your best bet. Depending on usage patterns, you could potentially handle that many on one server. If you mean one million all connecting at once, you probably need some horizontal scaling.

Go can probably provide 10–40k responses per second for typical server logic on good hardware. If your million users connect over the course of a few minutes, even the low end of that range could handle a million users, if it were even load, but Internet load tends to be fractal, so you'd probably want more capacity sitting idle most of the time to handle peak loads.

Reducing your need for backend servers is another strategy to bring to the table. Do as much work as possible on the client. Use a client framework like Angular, React, or Vue.js, so the browser is doing the template work. And for the love of everything good, put static files on a CDN. Yes, Go can serve a lot of files from one system, but you really want to use its capacity and bandwidth for something that isn't better done by a CDN.

If you have a server task that takes any amount of processing time (conversion of images or something), then you probably want to use a “serverless” approach like Amazon lambda. Then Amazon can handle the scaling for you.

As to databases: some scale horizontally better than others. Do your homework and figure out what database handles the access patterns you need. [2] Prescale your database servers to the level where they can support the access patterns you need. Auto scaling of large databases takes time, so don't expect to be able to spin up more capacity at the drop of a hat.

Or Google Cloud Storage [3] may be right for you; let Google handle the scaling, and you only pay for capacity and bandwidth.

Good luck.

Footnotes

First, when talking about databases in the Firebase suite, I’m only going to talk about Firestore, and not RTDB (Realtime Database, which was traditionally known as just Firebase). Firebase now refers to an entire suite of services, which includes Firestore, RTDB, file storage, authentication, etc. RTDB is most likely going to be phased out and Google is clearly putting their effort behind Firestore.

Second, there is no manual sharding in Firestore, like there was in RTDB. Sharding is handled automatically by Firestore, so ignore the other answers about that.

Third, you, the customer, own the da

First, when talking about databases in the Firebase suite, I’m only going to talk about Firestore, and not RTDB (Realtime Database, which was traditionally known as just Firebase). Firebase now refers to an entire suite of services, which includes Firestore, RTDB, file storage, authentication, etc. RTDB is most likely going to be phased out and Google is clearly putting their effort behind Firestore.

Second, there is no manual sharding in Firestore, like there was in RTDB. Sharding is handled automatically by Firestore, so ignore the other answers about that.

Third, you, the customer, own the data in your Firestore database, not Google, so ignore the other answers about that also. If Google, and not the customer, owned that data, Firebase would have virtually no users. And, also contrary to the other answers, you can export data from Firestore. If customers weren’t allowed to export their data, who would ever use it?

Fourth, the suggestion to use SQL technology on your own physical servers (ones that you physically own) is not something that I think belongs in an answer to this question. SQL vs NoSQL and cloud vs in-house are decisions that you should have already made if you’re asking about a specific serverless provider. Clearly (at least apparently), you’ve decided on serverless NoSQL, which is definitely the most popular choice today. And between the two most popular providers, AWS and Firebase, they are designed to handle virtually any workload you throw at it. The most highly-trafficked applications in the world run on NoSQL serverless, like the Amazon marketplace, for example (AWS, obviously). If AWS can handle it, Firebase can handle it too. Google would never design its NoSQL serverless to max out at a certain number of users or level of traffic; it’s designed to scale virtually infinitely.

The only caveat to Firestore (and NoSQL serverless in general, really) is dollar cost—right now in 2019, anyway. Firestore was considerably more expensive only a year ago than it is today, think about that. I think NoSQL serverless will undoubtedly continue to become cheaper, including Firestore. But, right now, with a relatively large userbase, NoSQL serverless, like Firestore, can get expensive. But so can doing it all in-house. An in-house solution still comes with bandwidth cost, in addition to hardware and human cost. Hiring qualified backend engineers to build and maintain an in-house setup is not cheap (hundreds of thousands of dollars a year). Firestore (and DynamoDB) have excellent price calculators to help you predict your costs and you may be surprised to see how cheap it is with a modestly-sized userbase, even with 100k users.

This is a question I get from a lot of clients as many of our products rely heavily on Firebase. The answer is very dependant on your application but in almost all cases this will never be a problem.

Where do the costs occur?

If you look at the Firebase pricing plan you can see that data is the main constraint for the pricing brackets. When you move up a tier you get various increases in data allowances as well as more database operations. This means you need to look at what your app does to see the impact of more users.

What does my app do?

This is the next question you need to ask yourself. Many

This is a question I get from a lot of clients as many of our products rely heavily on Firebase. The answer is very dependant on your application but in almost all cases this will never be a problem.

Where do the costs occur?

If you look at the Firebase pricing plan you can see that data is the main constraint for the pricing brackets. When you move up a tier you get various increases in data allowances as well as more database operations. This means you need to look at what your app does to see the impact of more users.

What does my app do?

This is the next question you need to ask yourself. Many apps will be dealing only in text strings meaning the data amount is very small. Images would be a step up from this and videos another jump in data usage. You need to think about how often your app stores this data. It might be that you only upload a single image for each user or it could be that you send multiple images while using. In both these cases the data usage will vary hugely.

Before worrying about how you will cope with 1M users think about how the app will fare with 1–10k users. In Firebase’s case you will probably be on the free tier (as long as you are not guzzling data) up to 100k users.

Think about this for a moment. You will have 100k users on your app. Even if you only add advertising you should be able to make the $25 a month to cover the Firebase FLAME costs.

The motto I use with my clients is this:

By the time you need to start worrying about backend costs for your app you can stop worrying about backend costs for your app

Once you reach the amount of users to requiring you to start payment you will have a successful app with a big user base which can be leveraged to start paying you back.

Should I use Firebase?

This is the final question you should ask yourself. Up until now this question could have been about one of the many different backends on offer.

If you are unsure I would be inclined to say: YES use Firebase!!

They are a great company and offer a fantastic product. Being owned by Google and having recently updated all their code they are unlikely to close down (RIP Parse). They also provide extensive and clear documentation and great support. There is a huge online community meaning sites like this and Stackoverflow have a huge number of questions to help you.

It is worth looking at your app though. Firebase does have some disadvantages:

  1. There is no easy way of querying data
  2. There is no way of executing code on the server

This is what should be researched when looking at whether Firebase is the right backend for you and even these can be mainly mitigated by using a separate server to control these tasks.

Conclusion: Unless your app is very data intensive then you won’t meet any of the data caps. Firebase will help you get your app online quickly and cheaply before scaling excellently once the users start pushing the upper limits.

NOTE: I would never recommend someone building their own backend when starting a new app. It might be that you have the skills to do it but in the long run it will cost you more (in your valuable time) than using another provider. Once your app is a great success and you are paying monthly fees to a backend then you can invest the money in creating a custom backend which suits your data type. Creating it yourself also requires a huge amount of invested time. If your app fails then this time is completely wasted.

There are several reasons not to avoid using Firebase and several reasons to avoid it altogether. Since you asked, the main reasons to avoid it are the following:

  • Outages: Firebase Realtime database seems to runs on a single zone and if it fails you cannot do anything but wait for it to recover and outages surely do happen.
  • Scaling: If your service grows a lot, you will start facing serious performance issues in reading and writing to the Realtime database.
  • Multiple environments: While Realtime has an emulator you can use for development, it comes with important limitations. The best solution see

There are several reasons not to avoid using Firebase and several reasons to avoid it altogether. Since you asked, the main reasons to avoid it are the following:

  • Outages: Firebase Realtime database seems to runs on a single zone and if it fails you cannot do anything but wait for it to recover and outages surely do happen.
  • Scaling: If your service grows a lot, you will start facing serious performance issues in reading and writing to the Realtime database.
  • Multiple environments: While Realtime has an emulator you can use for development, it comes with important limitations. The best solution seems to be to have a second project to use as a development/staging environment. Of course, if you want to have a few development environments plus a staging one before you roll out to production it starts to get messy. Being able to replicate your production environment using docker containers does make life and collaboration easier and you can’t do something like this with Firebase.

Nonetheless, I would not recommend just setting up your own backend and having to do software patches etc as some people imply is the alternative. The alternatives include using more scalable databases such as BigTable or ScyllaDB which can be used as a turnkey solution and then using things such as Cloud Functions, App Engine or Docker containers along with Kubernetes for your backend services. Just make sure that you encapsulate and abstract calls to your queues and databases so that you are not tied up in any one technology.

Of course, there are also very good reasons to use Firebase:

  • Authentication: Firebase takes care of your authentication allowing for different authentication methods that work on all devices. This makes life a lot easier.
  • On(change, add, delete etc.) Events: Realtime database is just amazing for frontend updates. You subscribe to data changes and the moment something changes in the database, your frontend updates instantly.
  • Cost: Especially in the beginning, it costs next to nothing to get started. As your app scales you start to feel the cost but in order to have a system that offers what Firebase offers you will be paying in any case so it is good to be able to start with very low fixed monthly costs and paying based on usage.
  • Speed of Development: Firebase allows you to launch your MVP in no time. It is not the best for large teams and complex projects but for MVPs it is just fantastic.

Well, I’d say the answer varies wildly for the exact use case you’re going for. So while most of the components that firebase offer to build apps are “planetary scale”, a lot of features are sorely lacking if you’re building an application on firebase features alone. Having built tens of projects, both as a hobby and as part of work I can give you a couple instances from my experience, but your mileage will vary depending on what you use firebase for.

  1. Hosting: firebase lends itself very easily to hosting thanks to the scale of Google’s infrastructure knowledge, built in CDN and SSL certificates

Well, I’d say the answer varies wildly for the exact use case you’re going for. So while most of the components that firebase offer to build apps are “planetary scale”, a lot of features are sorely lacking if you’re building an application on firebase features alone. Having built tens of projects, both as a hobby and as part of work I can give you a couple instances from my experience, but your mileage will vary depending on what you use firebase for.

  1. Hosting: firebase lends itself very easily to hosting thanks to the scale of Google’s infrastructure knowledge, built in CDN and SSL certificates and the new functions hosting that you can use for server side rendered node web apps.
  2. Fire store and real-time database: This is also massively scalable and you might find yourself using these databases even if you make the switch to other stacks just cause of the easy of use and the ability to have real-time listeners which is just such a breath of fresh air compared to MySQL or other databases. HOWEVER, this is also the part that I have the biggest issue with scalability. You just cannot do good queries with either of the databases. And this is a bigger problem than you realize and you will have to use an external system like Algolia, that has to index your database and something you have to pay extra for.
  3. Authentication: This just works out of the box and since they are effectively just wrappers around existing with providers like Facebook and google, they just work.
  4. Notifications: Having being built on Google’s Cloud Messaging system, this would also be infinitely scalable and should have no issues supporting a large app.
  5. Cloud functions: These work in a pinch when you need to run a couple functions in a server less environment, and to have an express app to host your website or to do certain tasks when data is written to or deleted from your database. HOWEVER, it has all the limitations of being a server less environment and doesn’t provide much control over its internal workings.

Having said all that, I am still an ardent firebase fan and I’ll continue using it for any projects I work on, whether at work or a personal project. And I am confident that over time, firebase will only get better with new features being added to it.

A Backend as a Service platform will make much easier the app development process. You will avoid unnecessary steps and reduce radically the go to the market time. Please see some benefits below:

  • Deploy apps quickly and predictably.
  • Roll out new app features seamlessly and reduce boiler plate code.
  • Scale your applications with no infrastructure hassles.

For full disclosure, I’m a founder at Back4app which is a BaaS platform. Back4App is an open source backend that uses Parse framework and helps developers build scalable and extensible applications at a much more rapid pace.

With Back4app the cost /

A Backend as a Service platform will make much easier the app development process. You will avoid unnecessary steps and reduce radically the go to the market time. Please see some benefits below:

  • Deploy apps quickly and predictably.
  • Roll out new app features seamlessly and reduce boiler plate code.
  • Scale your applications with no infrastructure hassles.

For full disclosure, I’m a founder at Back4app which is a BaaS platform. Back4App is an open source backend that uses Parse framework and helps developers build scalable and extensible applications at a much more rapid pace.

With Back4app the cost / API call for each app will reduce as the user base increases. For example, our entry level plan costs $4.99 and provides 500k requests. The cost for each 100,000 requests is $1.00. On the other hand our $99.99/month plan provides 20M requests and the cost for each 100,000 requests is $0.05. So, you pay 20x less if you use a high volume plan. This pretty inexpensive considering you don't have to handle servers and spend time writing boiler plate code.

Back4app is a fully managed backend platform comprising Automated Provisioning and Scaling of Parse Server Applications, App Migration, Web-Based Management Tools, Reliability, Backup and Recovery, 24*7 Monitoring and Alerting, Expert Support, and many other valuable features. One of the key merits of Back4App is that it allows you to customize and optimize each app differently, and this distinctive aspect makes it the most preferred choice among developers.

Choosing Back4App as your backend platform makes you forget the boilerplate code and infrastructure hassles and focus primarily on what truly matters to your Application. It enables you to provide your users the right features by adding custom modules in any technology or language of your choice. Whether your app needs a performative geo-query, a high-memory algorithm, any specific regulation security measure, or any particular npm module, Back4App has got you covered.

Our main features are:

· Push Notifications Tool: Allows you to easily setup your push notification mechanisms.

· Parse Server Dashboard: Helps you manage your app entities and push notification campaigns. Allows you to create, send, and segment push notification messages.

· Automatic E-mails: Empowers you to create, send, and segment push notification messages.

· CLPs (Class Level Permissions): Secures your app from the data browser with class level permissions.

· System E-mails: Allows you to set up automatic emails to user’s subscription and reset the password.

· Cloud Code Tool: Enables you to upload, debug, and run custom JavaScript code with Back4App.

· Background Jobs: Helps you schedule particular routines to run on your app background (Chron Jobs).

· Social Login: Assists you in configuring your app with Facebook and Twitter.

· Global Config: Empowers you to set specific variables to your app.

If you want to know more about backend as service, please access our website or check details of our product at A new Approach to the Backend as a Service Market – Back4app – Medium.

Yes but you will burn way too much money. Why?

Firebase is a general purpose very easy to use Api and so are the features. Very easy to use but not too rich of functionality. (you cannot get complex queries with Firebase e.g.)

You will easily build a Instagram Clone. However, at a certain point it’s simply too expensive and you’re better off developing your own stack to serve your needs efficiently.

Conclusion: Build an MVP with Firebase if you have no idea about backends. If your app gets traction you can easily transition to your own stack. (Most of the times you won’t get traction so there’s n

Yes but you will burn way too much money. Why?

Firebase is a general purpose very easy to use Api and so are the features. Very easy to use but not too rich of functionality. (you cannot get complex queries with Firebase e.g.)

You will easily build a Instagram Clone. However, at a certain point it’s simply too expensive and you’re better off developing your own stack to serve your needs efficiently.

Conclusion: Build an MVP with Firebase if you have no idea about backends. If your app gets traction you can easily transition to your own stack. (Most of the times you won’t get traction so there’s no need to waste time implementing your own stack)