• coffee_with_cream@sh.itjust.works
    link
    fedilink
    arrow-up
    6
    ·
    3 months ago

    For anyone doing a serious project, it’s much more cost effective to rent a node and run your own models on it. You can spin them up and down as needed, cache often-used queries, etc.

    • ☆ Yσɠƚԋσʂ ☆@lemmy.mlOP
      link
      fedilink
      arrow-up
      7
      arrow-down
      1
      ·
      3 months ago

      For sure, and in a lot of use cases you don’t even need a really big model. There are a few niche scenarios where you require a large context that’s not practical to run on your own infrastructure, but in most cases I agree.