K-WfGrid

K-WfGrid is an EU IST STREP project (*IST*-2002-511385). The goal of this project is to deliver a robust Grid environment which provides a user friendly interface for the composition of the Grid workflow applications, performs smart scheduling of application components and finally executes them on the available Grid resources. All phases of an application processing will be strongly supported by the domain specific knowledge and semantic technologies. The target applications could be from different scientific and business domains. For instance, scientific simulations like flood forecasting simulation and industrial applications like ERP and traffic management.


Besides taking part in project management, coordination and dissemination tasks, DPS group is also involved in the two technical work packages (WPs) of K-WfGrid, that is WP2 and WP3. These work packages target development of a scheduler and performance monitoring and analysis service, respectively. Fig. 1 depicts the workflows of the Grid services (see below).

Fig.1 K-Wf Grid system architecture.

Fig.2 Scheduler in K-Wf Grid system.


WP2: Workflow Orchestration and Execution Environment


Within WP2, DPS is developing a performance-driven scheduler. Scheduler is a service responsible for actual mapping of the workflow activities onto the Grid resources. It determines which instances of a set of alternative Web Service Operation (WSO) instances will be selected for invocation in the current workflow execution.
Scheduling decisions are supported by the knowledge collected in the general purpose knowledge-base, called GOM, and by the Performance Analysis and Monitoring service.

WP3: Performance Monitoring and Analysis for Grid Infrastructures and Applications

DPS is performing leading role in the work package 3. That means, along with research and development, DPS is coordinating the whole work conducted in WP3, as well.

DPS also focuses on studying important performance and dependability metrics along with their description, measurement and association with the Grid workflows. The performance of workflows will be examined through workflow-specific metrics. We introduce a novel ontology for performance data associated with the Grid workflows. This ontology describes well-defined concepts associated with Grid workflows and performance metrics associated with these concepts.

Our main aim is to develop a set of distributed performance analysis services (DIPAS). These services collect performance and monitoring data from the monitoring service and provide performance results to the client of the performance analysis service during the runtime of the workflows. DPS group aims at supporting performance interpretation and bottleneck search. It will develop a specification language for performance properties in combination with search processes for performance problems. We will also support interpretation of performance data.

DPS works on data representations and service interfaces. Performance, monitoring and event data can be represented in a well-defined XML format, thus interaction with other services is substantially simplified and becomes consistent. Instrumentation and measurement requests are issued dynamically via an XML interface.

Moreover, DPS also conducts research in developing adaptive monitoring for the Grid and instrumentation techniques for multilingual applications. This work is done with CYFRONET group in developing the Grid Monitoring and Instrumentation Service.

Posters and Demonstration movies

Posters

Demonstration movies

Software download


People involved

Past members:

For further information about DPS work on K-WfGrid please contact: Thomas Fahringer or Hong-Linh Truong.

Scientific contributions of DPS to K-Wfgrid

  1. Hong-Linh Truong, Schahram Dustdar, Thomas Fahringer "Performance Metrics and Ontology for Grid Workflows", (Submitted PDF), Future Generation of Computer Systems, (c) Elsevier, 2007. Accepted.

  2. Robert Samborski, Hong-Linh Truong, Thomas Fahringer,Monitoring and Analysis of Dependability Metrics of Grid Services (PDF Poster), Cracow Grid workshop, October 16-18, 2006, Cracow, Poland

  3. Francesco Nerieri, Radu Prodan, Thomas Fahringer,Kalipy: A Tool for Online Performance Analysis of Grid Workflows through Event Correlation(Submitted PDF),2nd IEEE International Conference on e-Science and Grid Computing, (c) IEEE Computer Society Press, Dec. 4- 6, 2006, Amsterdam, Netherlands.

  4. Hong-Linh Truong, Robert Samborski, Thomas Fahringer, Towards a Framework for Monitoring and Analyzing QoS Metrics of Grid Services (Submitted PDF), 2nd IEEE International Conference on e-Science and Grid Computing, (c) IEEE Computer Society Press, Dec. 4- 6, 2006, Amsterdam, Netherlands.

  5. Hong-Linh Truong, Peter Brunner, Thomas Fahringer, Francesco Nerieri, Robert Samborski, Bartosz Balis, Marian Bubak, Kuba Rozkwitalski, K-WfGrid Distributed Monitoring and Performance Analysis Services for Workflows in the Grid, (Submitted PDF), 2nd IEEE International Conference on e-Science and Grid Computing, (c) IEEE Computer Society Press, Dec. 4- 6, 2006, Amsterdam, Netherlands.

  6. Farrukh Nadeem, Muhammad Murtaza Yousaf, Radu Prodan, Thomas Fahringer, Soft Benchmarks-based Application Performance Prediction using a Minimum Training Set, (Submitted PDF), 2nd IEEE International Conference on e-Science and Grid Computing (e-Science 2006), (C) IEEE Computer Society Press, December 2006, Amsterdam, The Netherlands.

  7. Michael Mair, Jun Qin, Marek Wieczorek and Thomas Fahringer. Workflow Conversion and Processing in the ASKALON Grid Environment, 2nd Austrian Grid Symposium, September 2006, Innsbruck, Austria
  8. Marek Wieczorek, Mumtaz Siddiqui, Alex Villazon, Radu Prodan, Thomas Fahringer, Applying Advance Reservation to Increase Predictability of Workflow Execution on the Grid,(Submitted PDF), 2nd IEEE International Conference on e-Science and Grid Computing (e-Science 2006), (C) IEEE Computer Society Press, December 2006, Amsterdam, The Netherlands.

  9. Dieter Theiner, Marek Wieczorek, Reduction of Calibration Time of Distributed Hydrological Models by Use of Grid Computing and Nonlinear Optimisation Algorithms, 7th International Conference on Hydroinformatics (HIC 2006), September 2006, Nice, France

  10. Marek Wieczorek, Radu Prodan, Thomas Fahringer, Dynamic Programming Based Approach for Bicriteria Workflow Scheduling on the Grid, Poster on HPDC 2006.

  11. Bartosz Balis, Hong-Linh Truong, Marian Bubak, Thomas Fahringer,Krzysztof Guzy, Kuba Rozkwitalski, "An Instrumentation Infrastructure for Grid Workflow Applications", (Submitted PDF), International Symposium on Grid Computing, High-Performance and Distributed Applications (GADA06), LNCS, (c) Springer-Verlag, November 2-3, 2006, Montpellier, France.

  12. Peter Brunner, Hong-Linh Truong, Thomas Fahringer "Performance Monitoring and Visualization of Grid Scientific Workflows in ASKALON", (Submitted PDF), The 2nd International Conference on High Performance Computing and Communications (HPCC-06), LNCS, (c) Springer-Verlag, Munich, 13th-15th September, 2006.

  13. Francesco Nerieri, Radu Prodan, Thomas Fahringer, Hong-Linh Truong, " Performance Analysis of Grid Workflow Applications", (Submitted PDF),7th IEEE/ACM International Conference on Grid Computing (Grid'06), (C) IEEE Computer Society Press, September 28th-29th, 2006, Barcelona, Spain.

  14. Rubing Duan, Radu Prodan, Thomas Fahringer, Run-time Optimization for Grid Workflow Applications (Submitted PDF), 7th IEEE/ACM International Conference on Grid Computing (Grid'06), (C) IEEE Computer Society Press, September 2006, Barcelona, Spain.

  15. Rubing Duan, Radu Prodan, Thomas Fahringer, Data Mining-based Fault Prediction and Detection on the Grid (Submitted PDF), 15th IEEE International Symposium on High Performance Distributed Computing (HPDC'06), (C) IEEE Computer Society Press, June 2006, Paris, France.

  16. Thomas Fahringer, Radu Prodan, Juergen Hofer, Farrukh Nadeem, Francesco Nerieri, Stefan Podlipnig, Jun Qin, Duan Rubing, Mumtaz Siddiqui, Hong-Linh Truong, Alex Villazon, Marek Wieczorek, ASKALON: A Development and Grid Computing Environment for Scientific Workflows, In: Workflows for eScience, XXX (Eds.), Springer, 2006 (accepted).

  17. B. Balis, M. Bubak, J. Dziwisz, H.-L. Truong, and T. Fahringer, " Integrated Monitoring Framework for Grid Infrastructure and Applications", ( PDF), In P. Cunningham and M. Cunnin gham, editors, Innovation and the Knowledge Economy. Issues, Applications, Ca se Studies, pages 269-276, Ljubljana, Slovenia, October 2005. IOS Press.

  18. Marek Wieczorek, Radu Prodan, Thomas Fahringer, "Scheduling of Scientific Workflows in the ASKALON Grid Environment", ACM SIGMOD Record, 09/2005.

  19. Hong-Linh Truong, Bartosz Balis, Marian Bubak, Jakub Dziwisz, Thomas Fahringer, Andreas Hoheisel, "Towards Distributed Monitoring and Perfo rmance Analysis Services in the K-WfGrid Project", (PDF, PS.GZ), In Proceedings of 6th International Conference on Parallel Processing and Applied Mathematics (PPAM'20 05), LNCS, (c) Springer-Verlag, Poznan, Poland, 11-14 September, 2005.

  20. Marek Wieczorek, Radu Prodan, Thomas Fahringer, "Comparison of Workflow Scheduling Strategies on the Grid", In Proceedings of 6th International Conference on Parallel Processing and Applied Mathematics (PPAM 2005), LNCS 3911, ) Springer-Verlag.

  21. Hong-Linh Truong, Thomas Fahringer, Schahram Dustdar, "Dynamic Instrumentation, Performance Monitoring and Analysis of Grid Scientific Workflows", (PDF), 3(1-2):1-18, Journal of Grid Computing, (c) Springer Netherlands, June 2005.

  22. Hong-Linh Truong, Thomas Fahringer, "Soft Computing Approach to Performance Analysis of Parallel and Distributed Programs", (PDF), 11th International Euro-Par Conference (Euro-Par 2005), LNCS, (c) Springer-Verlag, Lisboa, Portugal, 30th August- 2nd September 2005.

  23. Hong-Linh Truong, Thomas Fahringer, Francesco Nerieri, Schahram Dustdar,"Performance Metrics and Ontology for Describing Performance Data of Grid Workflows", (PDF), IEEE International Symposium on Cluster Computing and Grid 2005 (CCGrid2005),1st International Workshop on Grid Performability, IEEE Computer Society Press, Cardiff, UK, 9 - 12 May 2005.

  24. Hong-Linh Truong, Thomas Fahringer, "Online Performance Monitoring and Analysis of Grid Scientific Workflows", (PDF), European Grid Conference 2005 (EGC2005), LNCS,(c) Springer-Verlag, Amsterdam, The Netherlands, February 14 -16, 2005.

Deliverables

  1. Hong-Linh Truong, Thomas Fahringer and Bartosz Balis, Performance Monitoring and Analysis Services - Stable Software, K-WfGrid D3 Deliverable, August, 2006.

  2. Hong-Linh Truong, Peter Brunner, Vlad Nae, Robert Samborski, Performance Analysis Service - Developer Manual, K-WfGrid Deliverable, October, 2006.
  3. Hong-Linh Truong, Robert Samborski, Peter Brunner Performance Analysis Service - User Manual, K-WfGrid Deliverable, February, 2007.

  4. Hong-Linh Truong, Performance Service Interfaces and Data Representation - Developer Manual, K-WfGrid Deliverable, September, 2006.

  5. Hong-Linh Truong, Performance Service Interfaces and Data Representation - User Manual, K-WfGrid Deliverable, September, 2006.

  6. Marek Wieczorek, Scheduler - User Manual, K-WfGrid Deliverable, September, 2006.
  7. Marek Wieczorek, Scheduler - Developer Manual, K-WfGrid Deliverable, September, 2006.
    8.
Last modified: by