How to run tensorflow models across private data without having extensive ETL pipelines in place? With Apache Wayang's Federated Deep Learning Platform Integration. We have added Tensorflow as a new platform inside the wayang-platforms parent module and implemented a Tensorflow driver. The TensorflowExecutor driver is responsible for creating and destroying Tensorflow resources, such as a model graph and a model parameter context. This way you will be able to run deep learning models with data feeds from your other platforms, like Apache Spark, Hadoop, Apache Flink or Apache Kafka directly integrated into #Tensorflow and TFF projects. Read more in your newest blog written by co-creator of Apache Wayang, Zoi Kaoudi: https://lnkd.in/eujExP2F
Apache Wayang
Software Development
San Jose, CA 29 followers
Next-gen Data Platform Integration
About us
Apache Wayang's three-layer architecture provides a strategic abstraction between user applications and underlying data processing platforms, ensuring seamless integration and optimization.
- Website
-
https://wayang.apache.org/
External link for Apache Wayang
- Industry
- Software Development
- Company size
- 11-50 employees
- Headquarters
- San Jose, CA
- Type
- Nonprofit
- Founded
- 2021
Locations
-
Primary
San Jose, CA, US
Updates
-
A new blogpost is up, written by Apache Wayang PPMC Gláucia Esppenchutz and it couldn't be more in time: Federated AI. LLM's struggle to show their business benefit. Why? Data access. AI needs the right data, in a suitable format at the right time. Apache Wayang does exactly that - providing LLM's and AI access with the right format (mostly JSON) to desperate data sources, like databases, files, specialized systems, most of them aren't even able to deal with data lakes. Read more in our blogpost: https://lnkd.in/eZuXcUuF #opensource #apachewayang #federatedlearning #AI #LLM
Wayang and the Federated AI | Apache Wayang (incubating)
wayang.apache.org
-
Our newest blogpost, written by Wayang developer Juri Petersen, explains how python works with Apache Wayang, and how we enable JSON conform UDF within a wayangPlan. The Python API allows users to specify WayangPlans with UDFs in Python. pywayangthen serializes the UDFs and constructs the WayangPlan in JSON format, preparing it to be sent to Apache Wayang's JSON API. Read more in our blogpost: https://lnkd.in/efMpPrVQ #python #federatedlearning #apachewayang
Pywayang - Apache Wayang's Python API | Apache Wayang (incubating)
wayang.apache.org
-
What are the differences between Wayang and Trino, aren't they the same? Read the blog written by Wayang PMC Zoi Kaoudi. Apache Wayang and Trino are two distinct systems that differ in their functionality and impact on different applications. Trino/Presto is a distributed SQL query engine with a coordinator and multiple workers, while Wayang is a middleware for integrating diverse data platforms. #presto #sqldeveloper #opensource #apachewayang https://lnkd.in/er2-5cCV
Apache Wayang vs. Presto/Trino | Apache Wayang (incubating)
wayang.apache.org
-
Part 2 of the blogposts written by Dr. Mirko Kämpf describes the implementation of a Kafka source and sink component in Apache Wayang for reading and writing data from and to Kafka topics. The post discusses three levels of implementation - an API level that exposes the functionality, a middleware level that handles wiring between abstractions and implementations, and an input/output connector level that implements the actual Kafka client logic. #apachewayang #apachekafka #datapipelines #etldeveloper https://lnkd.in/eP3X6ytK
Apache Kafka meets Apache Wayang - Part 2 | Apache Wayang (incubating)
wayang.apache.org
-
Read part 1 of the blog series, written by Dr. Mirko Kämpf from ecolytiq, about implementing a cross-organization decentralized data processing scenario. This blog series showcases Apache Wayang, a tool that enables collaborative data processing without centralizing it. Participants can keep their protected local data secure while working with public data from accessible sources. #apachewayang #dataprocessing #opensource #etldeveloper #datapipelines https://lnkd.in/eu2xvaKA
Apache Kafka meets Apache Wayang - Part 1 | Apache Wayang (incubating)
wayang.apache.org
-
We are happy to announce the release of Wayang 0.7.1, which comes with federated SQL and ML query optimization. #apachewayang #apachespark #apacheflink https://lnkd.in/esFbq3Er
[ANNOUNCE] Apache Wayang (Incubator) 0.7.1 released
mail-archive.com
-
The large variety of specialized data processing platforms and the increased complexity of data analytics has led to the need for unifying data analytics within a single framework. Such a framework should free users from the burden of (i) choosing the right platform(s) and~(ii) gluing code between the different parts of their pipelines. Apache Wayang (Incubating) is the only open-source framework that provides a systematic solution to unified data analytics by integrating multiple heterogeneous data processing platforms. #apachewayang #opensource #apachespark #apacheflink https://lnkd.in/ezc9iy8q
Apache Wayang: A Unified Data Analytics Framework | Kaustubh Beedkar
web.iitd.ac.in
-
Wayang is a powerful cross-platform middleware that can utilize and seamlessly integrate various execution platforms, including Postgres, Spark, Flink, Java Streams, and Python. In benchmarking, we found that Wayang demonstrated exceptional performance across multiple use cases, including a popular machine learning algorithm, a core big data benchmark task, and a relational query. #apachewayang #benchmarking #opensource https://lnkd.in/edqnKap7
Benchmarking Wayang | Apache Wayang (incubating)
wayang.apache.org
-
Apache Wayang at The Apache Software Foundation conference CommunityOverCode'23 #apachewayang #asf #opensource https://lnkd.in/eTr9Nt6n
Federated Cross-Platform Sql With Apache Wayang
https://www.youtube.com/