![]() your laptop or in this case either of CSC's supercomputers) fetches a workflow from the LaunchPad and executes it. Workflow (set of Fireworks including their dependencies and metadata)Ī FireWorker (e.g.FireTask (computing job to be performed).LaunchPad (manages workflows and metadata).The basic components of FireWorks are the Defining and executing workflows with FireWorks With this said, you can still continue to useĬonda environments, but only in case they are containerized. This causes startup delays andĭegrades the performance of Lustre for all users. For reference, a Conda installation of FireWorks contains more than 24000įiles, most of which are read each time the application is run. Installations on our supercomputers to avoid performance issues due to the large number of filesīrought by Conda. CSC has deprecated the direct usage of Conda ![]() Please do not install FireWorks in a Conda environment that is sitting directly on the shared Thus, you may need to separately specify the PyMongo version in the req.txt file when installing FireWorks. Note that the OpenShift template in Rahti sets up MongoDB version 3.2, requiring that the PyMongo version used with FireWorks cannot be newer than 3.12. The process of setting up and connecting to a MongoDB database in Rahti is detailed in a separate tutorial, see Accessing databases on Rahti from CSC supercomputers. To this end, you can use the -slim flag of pip-containerize to utilize a pre-built minimal Python container with a much newer version of Python than the system default 3.6.8. FireWorks requires at least Python 3.7, so make sure you're using at least this version. Note that the Python version used by pip-containerize is the first Python executable found in the path, so it's affected by loading modules. For further instructions, see the Tykky documentation. A plain pip installation with pip-containerize is enough, just add the line fireworks to the req.txt file containing the requirements of your environment. We recommend using Tykky to install FireWorks within a Singularity container. Installing FireWorks and setting up MongoDB in RahtiįireWorks is easy to install. Integrates with Slurm, but all subtasks must use identical resources.Supports complicated workflows with several dependent steps.Can handle parallel ( MPI/OpenMP) subtasks.This page describes how to define and execute FireWorks workflows in CSC's computing environment using a MongoDB running in the Rahti container cloud. Workflows are flexibly defined using YAML, JSON or through a Python API and stored in a MongoDB database. My project is in a private repo, but if you still need package.json files (there are actually 5, 1 in root + 4 packages I described above) let me know and I will publish them somewhere.FireWorks is a free, open-source tool for defining, managing and executing workflows with multiple steps and potentially complex dependencies. I was thinking, could any dependency mentioned above be conflicting with firebase-js-sdk in this setup? shared (constants mostly for re-use in other module, almost no dependencies)Īll of the 4 package dependencies get installed into the root node_modules so my project only uses 1 version of each dependency (I've double checked that firebase related dependencies are only in a root node_modules folder).backend ( firebase-admin + firebase-functions) react native app (uses react-native-firebase).react web app (uses this library, firebase-js-sdk),.I'm using firebase in a monorepo (managed by lerna) with 4 packages (each having it's own package.json): Will do my best to isolate the issue, but may not be easy because my setup is a bit complex.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |