ENGINEERING TECH

Prophecy Accelerates Adoption of Lakehouse Technology with Launch of "Prophecy for Databricks" as Demand for Modern Data Stack Skyrockets

Prophecy | June 23, 2022

Lakehouse Technology
Prophecy, the leading low-code platform for data engineering, announced the launch of Prophecy for Databricks, a powerful new offering that makes it easier and faster to build data pipelines that deliver the data for business intelligence and machine learning. This platform, with a visual drag-and-drop canvas, enables anyone that wants to do data engineering  to visually and interactively develop, deploy and monitor data pipelines on Apache Spark.

Built for use by both seasoned data engineering teams and non-programmer data citizens alike, Prophecy for Databricks enables many more users to build pipelines easily, move them to production and accelerate the transition of companies being data driven. With 10x users enabled, data teams experience a radical increase in operational excellence and data quality enabling them to manage more pipelines than ever before.

IDC has forecasted that data is being created at an annual growth rate of 23%, which mans 181 zettabytes of data will have been created by 2025. With data growing so quickly, corporations are struggling to keep up with processing the data at this pace. According to Gartner, the DBMS market is nearly $80B and grew 22% in just the last year, with the share of cloud DBMS growing even faster than the overall DBMS market.

Existing data engineering products do not meet the needs of companies and have proven to be unnecessarily complex and inefficient. With Prophecy for Databricks, companies can 10x data engineering with dramatic increases in data practitioners doing data engineering, individual productivity, reliability of data pipelines, and data quality.

The industry need for data & analytics far outstrips what can be produced by data engineers programming in notebooks. With this release of Prophecy for Databricks, we're providing powerful, visual tools that enable an order of magnitude more data users to quickly develop data pipelines, at the same level as programmers. This expansion of data engineering to non-programmers is the only way to realize the potential of data at scale."

Raj Bains, CEO and co-founder of Prophecy

With Prophecy for Databricks, companies can modernize their data pipelines on Spark through the platform's core features, which include:

  • A Visual Development Environment - An intuitive, low-code, drag-and-drop IDE enables all data practitioners, from non-programmer to expert, to develop data pipelines on Spark quickly and easily. The platform turns the visual data pipeline into 100% open-source Spark code (PySpark or Scala), with interactive development and execution to verify that pipeline works correctly every step of the way.
  • Productivity Enhancement - The ability to build and extend custom data frameworks in the visual elements, standardizes and reuses components, leading to improved efficiency, better collaboration, and reduces risk. GIT integration allows for tracking and versioning of changes, test coverage ensures all changes are unit tested, CI/CD moves changes from development to production with high confidence, and metadata search and lineage ensures data can be tracked all the way back to the source.
  • Seamless Integration - Prophecy for Databricks integrates smoothly with existing Databricks data stack utilized by enterprises. The technology is deployed within a company's existing Virtual Private Cloud (VPC) and integrates with all major data products and is extensible to support additional tools, including Delta Lake.
  1. Through Databricks Partner Connect, users can start using Prophecy directly within the Databricks User Interface (UI).
  2. The Prophecy for Databricks platform utilizes Databricks Workflows to simplify the orchestration and management of production workflows on any cloud, allowing for the running of code reliably in a Databricks cluster.

About Prophecy
Prophecy is the low-code data engineering company. Prophecy provides an easy-to-use visual interface to interactively build, deploy, execute, and manage data processing.  Using software engineering best practices with GIT, CI/CD, unit tests, and column-level lineage, Prophecy is trusted by enterprises including multiple companies in the Fortune 50 running thousands of data pipelines every day.

Spotlight

Robotic process automation (RPA) is an evolving technology that promises significant cost and productivity efficiencies. It automates repeatable tasks using a combination of AI and machine learning offering promising business values such as improved security, increased operational control, and reduced manual efforts. Hence, to enhance productivity, organizations and enterprises are planning to implement or are already in the process of implementing RPA solutions.

Spotlight

Robotic process automation (RPA) is an evolving technology that promises significant cost and productivity efficiencies. It automates repeatable tasks using a combination of AI and machine learning offering promising business values such as improved security, increased operational control, and reduced manual efforts. Hence, to enhance productivity, organizations and enterprises are planning to implement or are already in the process of implementing RPA solutions.

Related News

DESIGN

Sciens Building Solutions Acquires Central California Company

Sciens Building Solutions | July 15, 2022

Sciens Building Solutions ("Sciens") today announced the addition of Lodi, California-based Cen-Cal Fire Systems ("Cen-Cal") to its national portfolio of companies. Cen-Cal is the latest acquisition announcement from Sciens and adds to the company's footprint in the important and strategic Northern/Central California market, where it already has an established presence through its Bay Area Division, Sabah International. Cen-Cal will partner closely with Sabah in this region to deliver more value for their customers through cross-collaboration. "We chose to partner with Sciens to continue the success and growth of Cen-Cal by working alongside its experienced leadership team and leveraging the investment that they will provide to help our business grow while building on our great success, . Sciens' growing national presence, combined with Cen-Cal's established reputation in the Central Valley, positions us to develop even broader customer offerings and employee opportunities in the Northern/Central regions of California." -Byron Weisz, Vice President of Cen-Cal Fire Systems. In 1986, brothers Wayne and Byron Weisz founded Cen-Cal based on their extensive experience and knowledge in fire sprinkler design, fabrication, and installation. They have experience with wet, dry, pre-action and AFFF fire sprinkler systems in commercial, industrial, and multi-family residential applications. Cen-Cal has decades of experience in the specialty sprinkler and suppression markets and bring with them a long-standing respected reputation in the industry. "As we continue on our rapid growth trajectory, I'm pleased to welcome Cen-Cal and its very capable team to the fun Sciens journey, We have enjoyed significant success over the past five years with Sabah International in this region and are delighted to add the considerable sprinkler capabilities that Cen-Cal brings to the table. We have no plans of slowing down and continue to seek partnerships with companies like Cen-Cal that are closely aligned with our vision and core values." -Terry Heath, CEO Sciens'. About Sciens Building Solutions- Headquartered in San Francisco, California, and backed by The Carlyle Group, Sciens Building Solutions is a holding company focused on the fire detection and security services sector. This includes the design, installation, and provision of maintenance services for fire detection and clean agent fire suppression systems in a variety of commercial, institutional, and government facilities. About Cen-Cal Fire Systems- Cen-Cal was founded in 1986, when brothers Byron and Wayne Weisz combined their experience and knowledge in fire sprinkler design, fabrication, and installation to form the company with the business mantra of "doing what you tell people you are going to do." Cen-Cal's portfolio of expertise includes consultation, design, fabrication, installation, and service of underground and overhead fire protection systems. They have experience with wet, dry, pre-action and AFFF fire sprinkler systems in commercial, industrial, and multi-family residential applications.

Read More

ENGINEERING TECH

Audubon Engineering Company Enters a Three-year Contract With Shell Offshore Inc.

Audubon Engineering Company | June 03, 2022

Shell Offshore Inc., a subsidiary of Shell plc, has signed a three-year contract with Audubon Engineering Company LP (Audubon), a leading provider of engineering, procurement, construction, and fabrication services, to provide brownfield engineering and procurement (EP) services for Shell's Mars Corridor. The contract includes some of Shell's offshore assets in the U.S. Gulf of Mexico, including the Mars, Olympus, Ursa, and Vito tension leg platforms, and has two one-year extension options. The water depths in this deep-water portfolio range between 3,000 and 4,000 feet (914 to 1,200 meters). Topside engineering and procurement services are included in the contract, which includes single-well subsea tiebacks, crane, lifeboat, and HVAC replacements, controls, firewater system, and utility upgrades, gas-lift installation, and prefabricated skid packages. The contract will be carried out by Audubon's strong local operating centers in New Orleans, Louisiana, and Houston, Texas. This deal builds on the company's history of providing Shell with integrated engineering and technical services. We’re delighted that Shell has selected Audubon to be its contractor for these assets. We are committed to driving value and efficiency for Shell to further enhance and extend productivity across its assets in the Mars Corridor.” Ryan Hanemann, President of Audubon Engineering Company He further added, “Audubon is proud of our accomplishments in the GOM region, and this contract award further positions our business for growth and continued delivery in the area.”

Read More

MATERIALS,CONSTRUCTION

Carrier Completes Acquisition of Toshiba's Global Residential and Light Commercial HVAC Business

Carrier | August 01, 2022

Carrier Global Corporation (NYSE: CARR), the leading global provider of healthy, safe, sustainable and intelligent building and cold chain solutions, has completed its acquisition of substantially all of Toshiba Corporation's (TOKYO: 6502) ownership stake in Toshiba Carrier Corporation (TCC). A longstanding joint venture between Carrier and Toshiba, TCC is a global provider of residential and light commercial HVAC solutions, including VRF and heat pump products. "This acquisition presents tremendous growth opportunities for Carrier in the fast-growing, energy-efficient VRF and heat pump markets, Toshiba Carrier's differentiated technology, strong channel and global design and manufacturing footprint allow us to benefit from growing electrification and sustainability trends. We are thrilled to welcome the 6,600 talented TCC employees to the Carrier family." -David Gitlin, Chairman & CEO, Carrier. Building on Carrier's previous acquisition of Guangdong Giwee Group, TCC will join the newly created Global Comfort Solutions business unit within Carrier's HVAC segment. The new business will be headquartered in Tokyo under the leadership of Saif Siddiqui, a seasoned Carrier leader who will report directly to HVAC segment President Chris Nelson. Mr. Toru Kubo, who has served as President & CEO of TCC, will be joining the Global Comfort Solutions leadership team reporting to Siddiqui. Carrier acquired substantially all of Toshiba's interest in TCC for approximately $900 million, subject to customary closing adjustments. For the fiscal year 2021, TCC had annual sales of approximately $2 billion. About Carrier- As the leading global provider of healthy, safe, sustainable and intelligent building and cold chain solutions, Carrier Global Corporation is committed to making the world safer, sustainable and more comfortable for generations to come. From the beginning, we've led in inventing new technologies and entirely new industries. Today, we continue to lead because we have a world-class, diverse workforce that puts the customer at the center of everything we do.

Read More