3.5.6 Spark Master at spark://spark-master-0.spark-headless.data-engineering.svc.cluster.local:7077

  • URL: spark://spark-master-0.spark-headless.data-engineering.svc.cluster.local:7077
  • Alive Workers: 12
  • Cores in use: 24 Total, 24 Used
  • Memory in use: 48.0 GiB Total, 36.0 GiB Used
  • Resources in use:
  • Applications: 1 Running, 22 Completed
  • Drivers: 0 Running, 0 Completed
  • Status: ALIVE

Workers (19)

Worker IdAddressStateCoresMemoryResources
worker-20251216170633-192.168.26.130-38531 192.168.26.130:38531 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217001246-192.168.74.61-40055 192.168.74.61:40055 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217020439-192.168.58.201-42211 192.168.58.201:42211 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217200556-192.168.144.160-35471 192.168.144.160:35471 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217201805-192.168.152.226-42011 192.168.152.226:42011 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217201909-192.168.109.181-43331 192.168.109.181:43331 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217201947-192.168.50.11-46625 192.168.50.11:46625 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202033-192.168.109.182-43635 192.168.109.182:43635 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202113-192.168.119.206-41607 192.168.119.206:41607 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202204-192.168.119.207-44641 192.168.119.207:44641 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202253-192.168.119.210-46319 192.168.119.210:46319 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202337-192.168.144.169-38387 192.168.144.169:38387 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202414-192.168.152.236-43891 192.168.152.236:43891 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202457-192.168.144.171-42763 192.168.144.171:42763 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202727-192.168.206.193-40247 192.168.206.193:40247 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202727-192.168.206.194-42723 192.168.206.194:42723 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217202905-192.168.179.133-42469 192.168.179.133:42469 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217203105-192.168.74.73-39457 192.168.74.73:39457 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217203143-192.168.119.215-40303 192.168.119.215:40303 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)

Running Applications (1)

Application IDNameCoresMemory per ExecutorResources Per ExecutorSubmitted TimeUserStateDuration
app-20251217201950-0022
(kill)
ProjectSubmissionMediaDetailsIngestion 24 3.0 GiB 2025/12/17 20:19:50 spark RUNNING 13 min

Completed Applications (22)

Application IDNameCoresMemory per ExecutorResources Per ExecutorSubmitted TimeUserStateDuration
app-20251217201718-0021 DataLakeHouseTransfer 12 3.0 GiB 2025/12/17 20:17:18 spark FINISHED 1.9 min
app-20251217200459-0020 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 20:04:59 spark FINISHED 3.2 min
app-20251217200449-0019 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 20:04:49 spark FINISHED 2.7 min
app-20251217200350-0018 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 20:03:50 spark FINISHED 3.0 min
app-20251217200340-0017 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 20:03:40 spark FINISHED 1.4 min
app-20251217200339-0016 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 20:03:39 spark FINISHED 40 s
app-20251217152939-0015 DataLakeHouseTransfer 30 3.0 GiB 2025/12/17 15:29:39 spark FINISHED 23 min
app-20251217152919-0014 ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/17 15:29:19 spark FINISHED 19 min
app-20251217152551-0013 DataLakeHouseTransfer 10 3.0 GiB 2025/12/17 15:25:51 spark FINISHED 2.6 min
app-20251217150358-0012 DataLakeHouseTransfer 10 3.0 GiB 2025/12/17 15:03:58 spark FINISHED 2.9 min
app-20251217150320-0011 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 15:03:20 spark FINISHED 2.8 min
app-20251217150320-0010 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 15:03:20 spark FINISHED 2.3 min
app-20251217150258-0009 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 15:02:58 spark FINISHED 46 s
app-20251217150224-0008 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 15:02:24 spark FINISHED 27 s
app-20251217113600-0007 DataLakeHouseTransfer 30 3.0 GiB 2025/12/17 11:36:00 spark FINISHED 8.0 min
app-20251217111835-0006 ProjectSubmissionMediaDetailsIngestion 26 3.0 GiB 2025/12/17 11:18:35 spark FINISHED 21 min
app-20251217111534-0005 DataLakeHouseTransfer 10 3.0 GiB 2025/12/17 11:15:34 spark FINISHED 2.4 min
app-20251217110315-0004 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:03:15 spark FINISHED 3.7 min
app-20251217110259-0003 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:59 spark FINISHED 3.2 min
app-20251217110258-0002 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:58 spark FINISHED 2.5 min
app-20251217110257-0001 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:57 spark FINISHED 2.0 min
app-20251217110257-0000 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:57 spark FINISHED 1.7 min