Spark Connect server - Details for Stage 22 (Attempt 0)

Resource Profile Id: 0
Total Time Across All Tasks: 1 s
Locality Level Summary: Process local: 50
Input Size / Records: 11.5 KiB / 6
Shuffle Write Size / Records: 4.9 KiB / 50
Associated Job Ids: 15

digraph G { subgraph clusterstage_22 { label="Stage 22"; 37 [labelType="html" label="SQLExecutionRDD [37] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; subgraph cluster86 { label="Exchange"; 32 [labelType="html" label="ShuffledRowRDD [32] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster81 { label="WholeStageCodegen (2)"; 33 [labelType="html" label="MapPartitionsRDD [33] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster80 { label="DeserializeToObject"; 34 [labelType="html" label="MapPartitionsRDD [34] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster79 { label="MapPartitions"; 35 [labelType="html" label="MapPartitionsRDD [35] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster76 { label="WholeStageCodegen (3)"; 36 [labelType="html" label="MapPartitionsRDD [36] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster87 { label="map"; 38 [labelType="html" label="Delta Table State #0 - file:/home/jovyan/spark-warehouse/test.db/fhvhv_trip/_delta_log [38] [Cached] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster149 { label="WholeStageCodegen (1)"; 53 [labelType="html" label="MapPartitionsRDD [53] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select PULocationID, coun..."]; } subgraph cluster148 { label="ObjectHashAggregate"; 54 [labelType="html" label="MapPartitionsRDD [54] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select PULocationID, coun..."]; } subgraph cluster147 { label="Exchange"; 55 [labelType="html" label="MapPartitionsRDD [55] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select PULocationID, coun..."]; } } 32->33; 33->34; 34->35; 35->36; 36->37; 37->38; 38->53; 53->54; 54->55; }

31,32

Event Timeline

Enable zooming