Understanding Solar Energy

The company I work for (as a data engineer) does utility scale solar + battery installation and site management. We recently finished a large scale installation just outside of Las Vegas (by some measures the largest in the US). It was backed by a PE firm. Costs are getting so low, the tech so predictable, and with battery warranties around 20 years the PE firm is able to get pretty high return with a fairly low risk. They enter into a "power purchase agreement" with the utility so they know how much they will be able to sell the power for, and as long as we collect data on the batteries they will be able to be warrantied if there is an issue (but there rarely are issues).

The batteries are by far the most expensive portion of the setup. The solar by comparison is dirt cheap. We have single axis tracking like mentioned in the article. Every day we fully charge the batteries, and discharge them in the evening.

> I work for (as a data engineer) does utility scale solar + battery installation and site management.

Did you build your own excel/python nightmare or is everyone using 3rd party management software for this?

> as long as we collect data on the batteries they will be able to be warrantied

Can you share some of the data? Beyond power in/out, do you monitor humidity, vibrations, temperature ?

Our data pipeline looks like this:

hardware/PLC --modbus--> kepware --mqtt--> mosquito broker --mqtt--> mqtt2prometheustool --http--> Victoria Metrics

The mqtt2prometheustool is something we developed in house. I am looking at removing one or more of the above steps and using telegraf instead, as it can ingest OPCUA or modbus data directly.

We use excel files just as the output of our reporting tools. For analysis it's the standard python data science stack of pands/numpy/scipy. Most people work in Jupyter notebooks, and their tools are eventually moved to services in our k8s cluster.

Temp and voltage are the main "cell level" datapoints we collect. I don't think we have any vibration sensors at site now.

how do you like it? I have a 20 year career in large scale consumer app/web/b2c tech, but i've always wanted to work in renewables. Is it easy enough to break into? Is there many non-hardware roles (i have no hardware skills)? any advice / vibes?

It's a great job. I joined with no prior experience in the field, and none of the positions on my team require hardware experience.

I'm working in the IIoT domain too. Your workflow is interesting towards the end. Any particular reason, why you don't write it to some db like Timescale or Influx at the end without any prometheus conversion?

Does this include thermals on batteries? And how much power is spent on keeping them at the optimals? What about SoC/SoD figures?

Because without that the 20 year promise is bullshit.

I can sort of name ballpark figures for the above, the thing I can't get is how this can even approach profitable w/o hype and subsidies.

Victoria Metrics is a Prometheus compatible time series database and was being used before I joined the company. I haven’t had any issues with it so I didn’t see a reason to pull it from our stack. Are you saying Timescale or Influx can natively ingest MQTT messages?

As someone working in the IoT space - why pay for kepware for something that can be done in a few weeks by a developer? Telegraf or even a bit of programming will save a lot of pain

> Does this include thermals on batteries?

Yes, not just batteries, but we collect cell level temperature for all cells in all batteries.

> And how much power is spent on keeping them at the optimals?

We run an AC unit on every container to keep them cool. (Its in the NV desert so never any need for a heater)

> What about SoC/SoD figures?

We do compute estimates of SoC but as you probably know charge state isn't always easy to estimate. All we really know is voltage, c rate and time.

> I can sort of name ballpark figures for the above, the thing I can't get is how this can even approach profitable w/o hype and subsidies.

There is certainly risk involved with any investment. But when you are buying batteries at the scale we do the price is probably much lower than you are thinking. And if we do properly gather all the warranty data then the risk of loss on battery failure is minimized.

I am actively exploring replacing kepware with telegraf. It seems very promising. Kepware was purchased and deployed before I joined the company. I think it's the defacto standard in the controls world and they never really considered anything else.

Do you have experience with modbus in telegraf? If so I'd love to chat for a bit to learn what you've learned.

well.. okay.

NV looks like similar enough to mid-to-southern-africa where we did stuff.

an AC unit (6KW? 24KW?) (per what, TEU or double-TEU) doesn't look like something sustainable. but we had much less dense installs, so I'm not really ready to argue that

SoC vs thermals vs load/charge profile is very not-a-single-number, but when the battery banks suddenly start demanding replacement (+ african logistics ) one develops models for the monitoring dashboards quite fast indeed.

I still believe that 20 yr warranty is bullshit on any serious load cycle. But if the manufacturers are willing to swap them, then no problem of course.

It's just making mqtt messages from modbus is such a simple task that you can just write the software yourself. There are many intricate stuff in modbus that maybe does not translate too well for generic translators and where you want to have full control. For example messages over multiple registers, breaking interfaces between vender versions and all that legacy stuff.

Telegraf is also nice (but only used it for mqtt topics) but the same applies here. The functionality is fairly simple. In telegraf depending on your data your .conf file gets fairly large and has to be maintained. If you have your data model already in code it's fairly easy to just write it yourself and gain the simplicity of just using the classes you have anyway.

In my current stack the data ingestion both the initial data->mqtt and mqtt-> database/cloud is just small programs that share their internal data objects. It's very easy to maintain for a small team imo.

Is there any pros to use PostgreSQL for metrics in 2025?

Not the OTEL "telemetry" but sensors sending measurements from the field "telemetry". IT tech is, as usual, stealing perfectly defined engineering words and making them something else.

TimescaleDB is perfect if you also have relational data that you need to join with field data to the point that there no pros of using anything else for this use case, say you have 100000+ sensors and you need to group them by the customer site relations while aggregating per day statistics.

Similar to your mqtt converter you can make them ingest via Kafka or some other adapter in between. EMQX broker can directly write into TSDB but did not research into others whether they can or not.