Alternatives to n8n

Data science and engineering teams at Lyft maintain several big data pipelines that serve as the foundation for various types of analysis throughout the business.

Apache Airflow sits at the center of this big data infrastructure, allowing users to “programmatically author, schedule, and monitor data pipelines.” Airflow is an open source tool, and “Lyft is the very first Airflow adopter in production since the project was open sourced around three years ago.”

There are several key components of the architecture. A web UI allows users to view the status of their queries, along with an audit trail of any modifications the query. A metadata database stores things like job status and task instance status. A multi-process scheduler handles job requests, and triggers the executor to execute those tasks.

Airflow supports several executors, though Lyft uses CeleryExecutor to scale task execution in production. Airflow is deployed to three Amazon Auto Scaling Groups, with each associated with a celery queue.

Audit logs supplied to the web UI are powered by the existing Airflow audit logs as well as Flask signal.

Datadog, Statsd, Grafana, and PagerDuty are all used to monitor the Airflow system.

We are a young start-up with 2 developers and a team in India looking to choose our next ETL tool. We have a few processes in Azure Data Factory but are looking to switch to a better platform. We were debating Trifacta and Airflow. Or even staying with Azure Data Factory. The use case will be to feed data to front-end APIs.

Back in 2014, I was given an opportunity to re-architect SmartZip Analytics platform, and flagship product: SmartTargeting. This is a SaaS software helping real estate professionals keeping up with their prospects and leads in a given neighborhood/territory, finding out (thanks to predictive analytics) who's the most likely to list/sell their home, and running cross-channel marketing automation against them: direct mail, online ads, email... The company also does provide Data APIs to Enterprise customers.

I had inherited years and years of technical debt and I knew things had to change radically. The first enabler to this was to make use of the cloud and go with AWS, so we would stop re-inventing the wheel, and build around managed/scalable services.

For the SaaS product, we kept on working with Rails as this was what my team had the most knowledge in. We've however broken up the monolith and decoupled the front-end application from the backend thanks to the use of Rails API so we'd get independently scalable micro-services from now on.

Our various applications could now be deployed using AWS Elastic Beanstalk so we wouldn't waste any more efforts writing time-consuming Capistrano deployment scripts for instance. Combined with Docker so our application would run within its own container, independently from the underlying host configuration.

Storage-wise, we went with Amazon S3 and ditched any pre-existing local or network storage people used to deal with in our legacy systems. On the database side: Amazon RDS / MySQL initially. Ultimately migrated to Amazon RDS for Aurora / MySQL when it got released. Once again, here you need a managed service your cloud provider handles for you.

Future improvements / technology decisions included:

Caching: Amazon ElastiCache / Memcached CDN: Amazon CloudFront Systems Integration: Segment / Zapier Data-warehousing: Amazon Redshift BI: Amazon Quicksight / Superset Search: Elasticsearch / Amazon Elasticsearch Service / Algolia Monitoring: New Relic

As our usage grows, patterns changed, and/or our business needs evolved, my role as Engineering Manager then Director of Engineering was also to ensure my team kept on learning and innovating, while delivering on business value.

One of these innovations was to get ourselves into Serverless : Adopting AWS Lambda was a big step forward. At the time, only available for Node.js (Not Ruby ) but a great way to handle cost efficiency, unpredictable traffic, sudden bursts of traffic... Ultimately you want the whole chain of services involved in a call to be serverless, and that's when we've started leveraging Amazon DynamoDB on these projects so they'd be fully scalable.

When starting a new company and building a new product w/ limited engineering we chose to optimize for expertise and rapid development, landing on Rails API, w/ AngularJS on the front.

The reality is that we're building a CRUD app, so we considered going w/ vanilla Rails MVC to optimize velocity early on (it may not be sexy, but it gets the job done). Instead, we opted to split the codebase to allow for a richer front-end experience, focus on skill specificity when hiring, and give us the flexibility to be consumed by multiple clients in the future.

We also considered .NET core or Node.js for the API layer, and React on the front-end, but our experiences dealing with mature Node APIs and the rapid-fire changes that comes with state management in React-land put us off, given our level of experience with those tools.

We're using GitHub and Trello to track issues and projects, and a plethora of other tools to help the operational team, like Zapier, MailChimp, Google Drive with some basic Vue.js & HTML5 apps for smaller internal-facing web projects.

I've heard that I have the ability to write well, at times. When it flows, it flows. I decided to start blogging in 2013 on Blogger. I started a company and joined BizPark with the Microsoft Azure allotment. I created a WordPress blog and did a migration at some point. A lot happened in the time after that migration but I stopped coding and changed cities during tumultuous times that taught me many lessons concerning mental health and productivity. I eventually graduated from BizSpark and outgrew the credit allotment. That killed the WordPress blog.

I blogged about writing again on the existing Blogger blog but it didn't feel right. I looked at a few options where I wouldn't have to worry about hosting cost indefinitely and Jekyll stood out with GitHub Pages. The Importer was fairly straightforward for the existing blog posts.

Todo * Set up redirects for all posts on blogger. The URI format is different so a complete redirect wouldn't work. Although, there may be something in Jekyll that could manage the redirects. I did notice the old URLs were stored in the front matter. I'm working on a command-line Ruby gem for the current plan. * I did find some of the lost WordPress posts on archive.org that I downloaded with the waybackmachinedownloader. I think I might write an importer for that. * I still have a few Disqus comment threads to map

hello guys, I need your help. I created a website, I've been using Elementor forever, but yesterday I bought a template after I made the purchase I knew I made a mistake, cause the template was in HTML, can anyone please show me how to put this HTML template in my WordPress so it will be the face of my website, thank you in advance.

which of the ads platform pays better? What about PurpleAds?

Google AdSense has refused to post ads on my site.

Really can not decide which one to add. Google AdSense email say that they are ready to show ads... Taboola is on review.

As a small startup we are very conscious about picking up the tools we use to run the project. After suffering with a mess of using at the same time Trello , Slack , Telegram and what not, we arrived at a small set of tools that cover all our current needs. For product management, file sharing, team communication etc we chose Basecamp and couldn't be more happy about it. For Customer Support and Sales Intercom works amazingly well. We are using MailChimp for email marketing since over 4 years and it still covers all our needs. Then on payment side combination of Stripe and Octobat helps us to process all the payments and generate compliant invoices. On techie side we use Rollbar and GitLab (for both code and CI). For corporate email we picked G Suite. That all costs us in total around 300$ a month, which is quite okay.

When starting a new company and building a new product w/ limited engineering we chose to optimize for expertise and rapid development, landing on Rails API, w/ AngularJS on the front.

The reality is that we're building a CRUD app, so we considered going w/ vanilla Rails MVC to optimize velocity early on (it may not be sexy, but it gets the job done). Instead, we opted to split the codebase to allow for a richer front-end experience, focus on skill specificity when hiring, and give us the flexibility to be consumed by multiple clients in the future.

We also considered .NET core or Node.js for the API layer, and React on the front-end, but our experiences dealing with mature Node APIs and the rapid-fire changes that comes with state management in React-land put us off, given our level of experience with those tools.

We're using GitHub and Trello to track issues and projects, and a plethora of other tools to help the operational team, like Zapier, MailChimp, Google Drive with some basic Vue.js & HTML5 apps for smaller internal-facing web projects.

Looking for the best CRM choice for an early-stage tech company selling through product-led growth to medium and big companies. Don't know if Salesforce or HubSpot are too rigid for PGL and expensive. I also had an experience of companies outgrowing Pipedrive pretty fast

Comparing HubSpot and Freshsales, not sure which to choose. Company and contact information is shareable among tech and sales teams allowing both parties to upkeep customers' contact details. Capturing leads from social media and system assigning to sales or having the option to manual assign. Sales follow up with sales activities. Once deal, technical involve to follow up regular customer visits, support ticketing, training, remind customers to renew licenses, work on projects and etc. Require a single platform to share a calendar to understand internal team activities and customer activities.

Hi, I am working as a web developer (PHP, Laravel, AngularJS, and MySQL) with more than 8 years of experience and looking for a tech stack that pays better. I have a little bit of knowledge of Core Java. For better opportunities, Should I learn Java, Spring Boot or Python. Or should I learn Drupal, WordPress or Magento? Any guidance would be really appreciated! Thanks.

Depends on what options and technologies you have available, and how do you deploy your website.

There are CMSs which update existing static pages through FTP: You provide access credentials, mark editable parts of your HTML in a markup, and then edit the content through the hosted CMS. I know two systems which work like that: Cushy CMS and Surreal CMS.

If the source of your site is versioned through Git (and hosted on GitHub), you have other options, like Netlify CMS, Spinal CMS, Siteleaf, Forestry, or CloudCannon. Some of these also need you to use static site generator (like 11ty, Jekyll, or Hugo).

If you have some server-side scripting support available (typically PHP) you can also consider some flat-file based, server-side systems, like Kirby CMS or Lektor, which are usually simpler to retrofit into an existing template than “traditional” CMSs (WordPress, Drupal).

Finally, you could also use a desktop-based static site generator which provides a user-friendly GUI, and then locally generates and uploads the website. For example Publii, YouDoCMS, Agit CMS.