Gourav Shivhare Tech

ELT Process Extraction, Load and Transform (ELT) is the technique of extracting raw data from the source and storing it in data warehouse of the target server and preparing it for endstream users. ELT comprises of 3 different operations performed on the data – Extract Extracting data is the technique of identifying data from one or more sources. The sources may be databases, files, ERP, CRM or any other useful source of data. Load Loading is the process of storing the extracted raw data in data warehouse or data lakes. Transform Data transformation is the process in which the raw data source is transformed to the target format required for analysis. Data from the sources are extracted and stored in the data warehouse. The entire data is not transformed but only the required transformation is done when necessary. Raw data can be retrieved from the warehouse anytime when required. The data transformed as required is then sent forward for analysis. When you use ELT, you move the entire da...

MuleSoft's Anypoint Runtime Fabric (RTF) has many features that help with deployment and management of Mule applications: Deployment: RTF can deploy applications to any environment, including on-premises, in the cloud, or in a hybrid setup. It can also automatically deploy Mule runtimes into containers. Isolation: RTF can isolate applications by running a separate Mule runtime server for each application. Scaling: RTF can scale applications across multiple replicas. Fail-over: RTF can automatically fail over applications. Monitoring and logging: RTF has built-in monitoring and logging capabilities to help teams troubleshoot issues and gain insights into application performance. Containerization: RTF supports containerization, which allows applications to be packaged with their dependencies and run consistently across different environments. Integration: RTF can integrate with services like SaveMyLeads to automate data flow between applications. Management: RTF can be managed with A...

Service Mesh A service mesh is an architectural pattern for microservices deployments . It’s primary goal is to make service-to-service communications secure, fast, and reliable. In a service mesh architecture, microservices within a given deployment or cluster interact with each other through sidecar proxy’s. The security and communication rules behind these interactions are directed through a control plane. The developer can configure and add policies at the control plane level, and abstract the governance considerations behind microservices, regardless of the technology used to build. Popular Service Mesh frameworks, such as Istio, have emerged to help organizations implement these architectural patterns.  A service mesh is a dedicated infrastructure layer that controls service-to-service communication within a distributed application. This method enables separate parts of an application to communicate with each other. Service meshes appear commonly in concert with cloud-native ...

What is Jenkins? Jenkins is an easy-to-use open-source java-based CI/CD tool. Jenkins has huge community support and an ocean of plugins that can integrate with many open-source and enterprise tools to make your life so easy. The following diagram shows the overall architecture of Jenkins and the connectivity workflow. Following are the key components in Jenkins Jenkins Master Node Jenkins Agent Nodes/Clouds Jenkins Web Interface Jenkins Server (Formerly Master) Jenkins’s server or master node holds all key configurations. Jenkins master server is like a control server that orchestrates all the workflow defined in the pipelines. For example, scheduling a job, monitoring the jobs, etc. Let’s have a look at the key Jenkins master components. Jenkins Jobs A job is a collection of steps that you can use to build your source code, test your code, run a shell script, run an Ansible role in a remote host or execute a terraform play, etc. We normally call it a Jenkins pipeline . Jenkins JOB :...

RabbitMQ , known for flexibility and support for multiple messaging protocols, is ideal for complex routing, message prioritization, and reliable delivery in microservices architectures. However, it may not handle high-throughput scenarios as efficiently as Kafka.   On the other hand, Apache Kafka , a distributed streaming platform, excels in high-throughput and fault-tolerant messaging, making it perfect for real-time data processing and analytics. Kafka typically outperforms RabbitMQ and ActiveMQ in high-volume scenarios due to its distributed nature. ActiveMQ , a Java-based message broker supporting the Java Message Service API, is favored in traditional enterprise messaging scenarios requiring robust security and transactional messaging. While offering good performance and enterprise features, ActiveMQ may not match Kafka's throughput capabilities in streaming scenarios. Each platform has its strengths and best-fit scenarios. Understanding their architectures and performance c...

To achieve optimal performance from your Mule applications, you must evaluate both the applications themselves and the environment in which they run. Although Mule 4 is designed to tune itself, your applications might exhibit performance issues due to their initial construction or dependencies. Similarly, for on-premises installations, you might need to tune the environment itself so that your Mule applications can take full advantage of it. Because many variables influence it, tuning the performance of your application requires some trial and error. You can simplify performance tuning by using documented best practices and testing your applications in ideal test environments. The following recommendations come from the Development and Services Engineering teams and benchmarking efforts by MuleSoft Performance Engineering. Optimizing the performance of your Mule apps requires the following actions: Applying tuning recommendations at the application level        ...