Spark Connect server - Details for Stage 27 (Attempt 0)

Resource Profile Id: 0
Total Time Across All Tasks: 0.3 s
Locality Level Summary: Process local: 50
Input Size / Records: 11.5 KiB / 6
Associated Job Ids: 17

digraph G { subgraph clusterstage_27 { label="Stage 27"; 37 [labelType="html" label="SQLExecutionRDD [37] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; subgraph cluster86 { label="Exchange"; 32 [labelType="html" label="ShuffledRowRDD [32] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster81 { label="WholeStageCodegen (2)"; 33 [labelType="html" label="MapPartitionsRDD [33] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster80 { label="DeserializeToObject"; 34 [labelType="html" label="MapPartitionsRDD [34] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster79 { label="MapPartitions"; 35 [labelType="html" label="MapPartitionsRDD [35] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster76 { label="WholeStageCodegen (3)"; 36 [labelType="html" label="MapPartitionsRDD [36] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster87 { label="map"; 38 [labelType="html" label="Delta Table State #0 - file:/home/jovyan/spark-warehouse/test.db/fhvhv_trip/_delta_log [38] [Cached] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select count(*) from test..."]; } subgraph cluster164 { label="WholeStageCodegen (1)"; 59 [labelType="html" label="MapPartitionsRDD [59] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select PULocationID, coun..."]; } subgraph cluster172 { label="mapPartitionsInternal"; 60 [labelType="html" label="MapPartitionsRDD [60] [Unordered] Spark Connect - session_id: "58d45445-3af3-4104-9739-56b4041ec0f9"\nplan {\n command {\n sql_command {\n sql: "select PULocationID, coun..."]; } } 32->33; 33->34; 34->35; 35->36; 36->37; 37->38; 38->59; 59->60; }

31,32

Event Timeline

Enable zooming