3.5.6 Spark Master at spark://spark-master-0.spark-headless.data-engineering.svc.cluster.local:7077

  • URL: spark://spark-master-0.spark-headless.data-engineering.svc.cluster.local:7077
  • Alive Workers: 15
  • Cores in use: 30 Total, 30 Used
  • Memory in use: 60.0 GiB Total, 45.0 GiB Used
  • Resources in use:
  • Applications: 2 Running, 114 Completed
  • Drivers: 0 Running, 0 Completed
  • Status: ALIVE

Workers (18)

Worker IdAddressStateCoresMemoryResources
worker-20251217001246-192.168.74.61-40055 192.168.74.61:40055 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251217020439-192.168.58.201-42211 192.168.58.201:42211 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218011200-192.168.212.232-45105 192.168.212.232:45105 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218152528-192.168.245.245-37619 192.168.245.245:37619 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218152609-192.168.115.137-40745 192.168.115.137:40745 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218152759-192.168.70.160-35439 192.168.70.160:35439 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218152841-192.168.69.122-42965 192.168.69.122:42965 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218152922-192.168.245.242-43125 192.168.245.242:43125 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153003-192.168.69.124-43571 192.168.69.124:43571 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153138-192.168.4.197-34709 192.168.4.197:34709 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153258-192.168.61.198-46313 192.168.61.198:46313 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153331-192.168.129.3-45291 192.168.129.3:45291 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153425-192.168.115.144-43589 192.168.115.144:43589 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153510-192.168.61.208-39941 192.168.61.208:39941 DEAD 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153552-192.168.4.200-42555 192.168.4.200:42555 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218153639-192.168.115.146-40989 192.168.115.146:40989 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218154419-192.168.74.1-42461 192.168.74.1:42461 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)
worker-20251218154553-192.168.162.56-44075 192.168.162.56:44075 ALIVE 2 (2 Used) 4.0 GiB (3.0 GiB Used)

Running Applications (2)

Application IDNameCoresMemory per ExecutorResources Per ExecutorSubmitted TimeUserStateDuration
app-20251218154109-0115
(kill)
DataLakeHouseTransfer 0 3.0 GiB 2025/12/18 15:41:09 spark WAITING 4.9 min
app-20251218152707-0114
(kill)
ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/18 15:27:07 spark RUNNING 19 min

Completed Applications (114)

Application IDNameCoresMemory per ExecutorResources Per ExecutorSubmitted TimeUserStateDuration
app-20251218152349-0113 DataLakeHouseTransfer 10 3.0 GiB 2025/12/18 15:23:49 spark FINISHED 2.5 min
app-20251218150602-0112 DataLakeHouseTransfer 12 3.0 GiB 2025/12/18 15:06:02 spark FINISHED 1.2 min
app-20251218150315-0111 DataLakeHouseTransfer 10 3.0 GiB 2025/12/18 15:03:15 spark FINISHED 3.2 min
app-20251218150244-0110 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 15:02:44 spark FINISHED 1.5 min
app-20251218150237-0109 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 15:02:37 spark FINISHED 1.1 min
app-20251218150233-0108 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 15:02:33 spark FINISHED 30 s
app-20251218113054-0107 DataLakeHouseTransfer 28 3.0 GiB 2025/12/18 11:30:54 spark FINISHED 17 min
app-20251218112258-0106 ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/18 11:22:58 spark FINISHED 20 min
app-20251218111938-0105 DataLakeHouseTransfer 10 3.0 GiB 2025/12/18 11:19:38 spark FINISHED 2.6 min
app-20251218110310-0104 DataLakeHouseTransfer 10 3.0 GiB 2025/12/18 11:03:10 spark FINISHED 3.4 min
app-20251218110308-0103 DataLakeHouseTransfer 10 3.0 GiB 2025/12/18 11:03:08 spark FINISHED 2.7 min
app-20251218110306-0102 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 11:03:06 spark FINISHED 1.3 min
app-20251218110306-0101 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 11:03:06 spark FINISHED 36 s
app-20251218110239-0100 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 11:02:39 spark FINISHED 37 s
app-20251218083947-0099 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 08:39:47 spark FINISHED 8.0 min
app-20251218082118-0098 ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/18 08:21:18 spark FINISHED 22 min
app-20251218081809-0097 DataLakeHouseTransfer 8 3.0 GiB 2025/12/18 08:18:09 spark FINISHED 2.5 min
app-20251218080241-0096 DataLakeHouseTransfer 8 3.0 GiB 2025/12/18 08:02:41 spark FINISHED 3.0 min
app-20251218080239-0095 DataLakeHouseTransfer 8 3.0 GiB 2025/12/18 08:02:39 spark FINISHED 2.4 min
app-20251218080239-0094 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 08:02:39 spark FINISHED 55 s
app-20251218080211-0093 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 08:02:11 spark FINISHED 44 s
app-20251218080149-0092 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 08:01:49 spark FINISHED 31 s
app-20251218053417-0091 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 05:34:17 spark FINISHED 8.4 min
app-20251218051942-0090 ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/18 05:19:42 spark FINISHED 19 min
app-20251218051630-0089 DataLakeHouseTransfer 10 3.0 GiB 2025/12/18 05:16:30 spark FINISHED 2.5 min
app-20251218050337-0088 DataLakeHouseTransfer 8 3.0 GiB 2025/12/18 05:03:37 spark FINISHED 2.9 min
app-20251218050336-0087 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 05:03:36 spark FINISHED 2.2 min
app-20251218050251-0086 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 05:02:51 spark FINISHED 2.4 min
app-20251218050250-0085 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 05:02:50 spark FINISHED 1.8 min
app-20251218050233-0084 DataLakeHouseTransfer 6 3.0 GiB 2025/12/18 05:02:33 spark FINISHED 1.7 min
app-20251218012139-0083 DataLakeHouseTransfer 18 3.0 GiB 2025/12/18 01:21:39 spark FINISHED 3.4 min
app-20251218011002-0082 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 01:10:02 spark FINISHED 2.8 min
app-20251218010954-0081 DataLakeHouseTransfer 28 3.0 GiB 2025/12/18 01:09:54 spark FINISHED 2.0 min
app-20251218010953-0080 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 01:09:53 spark FINISHED 1.4 min
app-20251218010921-0079 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 01:09:21 spark FINISHED 1.3 min
app-20251218010620-0078 MonolithReleasesExportJob 30 4.0 GiB 2025/12/18 01:06:20 root FINISHED 3.7 min
app-20251218010603-0077 MonolithMediaAssignmentMediaLogExportJob 30 4.0 GiB 2025/12/18 01:06:03 root FINISHED 3.5 min
app-20251218010603-0076 MonolithUserEventsExportJob 30 4.0 GiB 2025/12/18 01:06:03 root FINISHED 2.5 min
app-20251218010334-0075 DataLakeHouseTransfer-ProjectUsers 30 3.0 GiB 2025/12/18 01:03:34 spark FINISHED 4.2 min
app-20251218010333-0074 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 01:03:33 spark FINISHED 3.3 min
app-20251218010328-0073 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 01:03:28 spark FINISHED 2.7 min
app-20251218010325-0072 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 01:03:25 spark FINISHED 2.3 min
app-20251218010312-0071 MonolithMediaCategoriesExportJob 30 4.0 GiB 2025/12/18 01:03:12 root FINISHED 1.8 min
app-20251218010310-0070 MonolithCategoriesExportJob 30 4.0 GiB 2025/12/18 01:03:10 root FINISHED 1.4 min
app-20251218010308-0069 MonolithCompaniesExportJob 30 4.0 GiB 2025/12/18 01:03:08 root FINISHED 59 s
app-20251218004141-0068 DataLakeHouseTransfer 30 1024.0 MiB 2025/12/18 00:41:41 spark FINISHED 21 min
app-20251218003808-0067 MonolithAgenciesExportJob 30 4.0 GiB 2025/12/18 00:38:08 root FINISHED 23 min
app-20251218003808-0066 MonolithSalesExportJob 30 4.0 GiB 2025/12/18 00:38:08 root FINISHED 23 min
app-20251218003309-0065 DataLakeHouseTransfer 30 3.0 GiB 2025/12/18 00:33:09 spark FINISHED 27 min
app-20251218003030-0064 ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/18 00:30:30 spark FINISHED 25 min
app-20251218002906-0063 DataLakeHouseTransfer 30 1024.0 MiB 2025/12/18 00:29:06 spark FINISHED 8.3 min
app-20251218002856-0062 DataLakeHouseTransfer 30 2.0 GiB 2025/12/18 00:28:56 spark FINISHED 7.3 min
app-20251218002518-0061 MonolithUsersExportJob 30 4.0 GiB 2025/12/18 00:25:18 root FINISHED 9.6 min
app-20251218002517-0060 MonolithUsersUserDetailsExportJob 30 4.0 GiB 2025/12/18 00:25:17 root FINISHED 8.8 min
app-20251218001948-0059 DataLakeHouseTransfer 22 3.0 GiB 2025/12/18 00:19:48 spark FINISHED 10.0 min
app-20251218001046-0058 AccountingLedgerEventsLoad 20 3.0 GiB 2025/12/18 00:10:46 spark FINISHED 18 min
app-20251218000718-0057 DataLakeHouseTransfer 18 3.0 GiB 2025/12/18 00:07:18 spark FINISHED 20 min
app-20251218000141-0056 DataLakeHouseTransfer 14 1024.0 MiB 2025/12/18 00:01:41 spark FINISHED 25 min
app-20251218000131-0055 DataLakeHouseTransfer 14 3.0 GiB 2025/12/18 00:01:31 spark FINISHED 25 min
app-20251218000130-0054 DataLakeHouseTransfer 14 3.0 GiB 2025/12/18 00:01:30 spark FINISHED 24 min
app-20251217235027-0053 DataLakeHouseTransfer 10 2.0 GiB 2025/12/17 23:50:27 spark FINISHED 34 min
app-20251217234405-0052 MonolithMediaAgenciesMediaExportJob 10 4.0 GiB 2025/12/17 23:44:05 root FINISHED 39 min
app-20251217234205-0051 DataLakeHouseTransfer 26 2.0 GiB 2025/12/17 23:42:05 spark FINISHED 1.6 min
app-20251217233908-0050 MonolithMediaDetailsExportJob 26 4.0 GiB 2025/12/17 23:39:08 root FINISHED 3.7 min
app-20251217233905-0049 MonolithEthnicitiesExportJob 26 4.0 GiB 2025/12/17 23:39:05 root FINISHED 49 s
app-20251217233559-0048 DataLakeHouseTransfer 22 2.0 GiB 2025/12/17 23:35:59 spark FINISHED 2.8 min
app-20251217233308-0047 MonolithPropertyReleasesExportJob 20 4.0 GiB 2025/12/17 23:33:08 root FINISHED 4.9 min
app-20251217233306-0046 MonolithMediaReleaseExportJob 16 4.0 GiB 2025/12/17 23:33:06 root FINISHED 1.9 min
app-20251217233304-0045 MonolithModelReleasesExportJob 18 4.0 GiB 2025/12/17 23:33:04 root FINISHED 1.5 min
app-20251217230248-0044 DataLakeHouseTransfer 14 2.0 GiB 2025/12/17 23:02:48 spark FINISHED 30 min
app-20251217230103-0043 MonolithMediaExportJob 10 4.0 GiB 2025/12/17 23:01:03 root FINISHED 31 min
app-20251217230049-0042 MonolithKeywordsExportJob 10 4.0 GiB 2025/12/17 23:00:49 root FINISHED 25 min
app-20251217225356-0041 DataLakeHouseTransfer 24 2.0 GiB 2025/12/17 22:53:56 spark FINISHED 1.1 min
app-20251217225010-0040 DataLakeHouseTransfer 16 2.0 GiB 2025/12/17 22:50:10 spark FINISHED 41 s
app-20251217224735-0039 DataLakeHouseTransfer 10 2.0 GiB 2025/12/17 22:47:35 spark FINISHED 1.1 min
app-20251217223833-0038 DataLakeHouseTransfer 10 2.0 GiB 2025/12/17 22:38:33 spark FINISHED 1.2 min
app-20251217222920-0037 DataLakeHouseTransfer 20 2.0 GiB 2025/12/17 22:29:20 spark FINISHED 1.1 min
app-20251217222400-0036 DataLakeHouseTransfer 18 2.0 GiB 2025/12/17 22:24:00 spark FINISHED 39 s
app-20251217221945-0035 DataLakeHouseTransfer 10 2.0 GiB 2025/12/17 22:19:45 spark FINISHED 1.4 min
app-20251217221034-0034 DataLakeHouseTransfer 10 2.0 GiB 2025/12/17 22:10:34 spark FINISHED 35 s
app-20251217220734-0033 DataLakeHouseTransfer 6 2.0 GiB 2025/12/17 22:07:34 spark FINISHED 1.0 min
app-20251217212743-0032 AgenciesChannelManagerAgencyMediaExportJob 22 4.0 GiB 2025/12/17 21:27:43 root FINISHED 1.2 min
app-20251217212447-0031 AgenciesChannelManagerAgencyMediaExportJob 22 4.0 GiB 2025/12/17 21:24:47 root FINISHED 2.1 min
app-20251217212103-0030 AgenciesChannelManagerAgencyMediaMainExportJob 16 4.0 GiB 2025/12/17 21:21:03 root FINISHED 2.6 min
app-20251217210610-0029 DataLakeHouseTransfer 14 3.0 GiB 2025/12/17 21:06:10 spark FINISHED 16 min
app-20251217210549-0028 sequential-editing3-mapping-load 12 3.0 GiB 2025/12/17 21:05:49 spark FINISHED 15 min
app-20251217210351-0027 AgenciesChannelManagerAgencyMediaExportJob 12 4.0 GiB 2025/12/17 21:03:51 root FINISHED 16 min
app-20251217210253-0026 data-program-exports-load 8 3.0 GiB 2025/12/17 21:02:53 spark FINISHED 45 s
app-20251217210104-0025 AgenciesChannelManagerSaleExportJob 8 4.0 GiB 2025/12/17 21:01:04 root FINISHED 1.5 min
app-20251217210033-0024 exports-google-sheet 6 1024.0 MiB 2025/12/17 21:00:33 spark FINISHED 41 s
app-20251217204320-0023 DataLakeHouseTransfer 30 3.0 GiB 2025/12/17 20:43:20 spark FINISHED 3.8 min
app-20251217201950-0022 ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/17 20:19:50 spark FINISHED 23 min
app-20251217201718-0021 DataLakeHouseTransfer 12 3.0 GiB 2025/12/17 20:17:18 spark FINISHED 1.9 min
app-20251217200459-0020 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 20:04:59 spark FINISHED 3.2 min
app-20251217200449-0019 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 20:04:49 spark FINISHED 2.7 min
app-20251217200350-0018 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 20:03:50 spark FINISHED 3.0 min
app-20251217200340-0017 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 20:03:40 spark FINISHED 1.4 min
app-20251217200339-0016 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 20:03:39 spark FINISHED 40 s
app-20251217152939-0015 DataLakeHouseTransfer 30 3.0 GiB 2025/12/17 15:29:39 spark FINISHED 23 min
app-20251217152919-0014 ProjectSubmissionMediaDetailsIngestion 30 3.0 GiB 2025/12/17 15:29:19 spark FINISHED 19 min
app-20251217152551-0013 DataLakeHouseTransfer 10 3.0 GiB 2025/12/17 15:25:51 spark FINISHED 2.6 min
app-20251217150358-0012 DataLakeHouseTransfer 10 3.0 GiB 2025/12/17 15:03:58 spark FINISHED 2.9 min
app-20251217150320-0011 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 15:03:20 spark FINISHED 2.8 min
app-20251217150320-0010 DataLakeHouseTransfer 8 3.0 GiB 2025/12/17 15:03:20 spark FINISHED 2.3 min
app-20251217150258-0009 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 15:02:58 spark FINISHED 46 s
app-20251217150224-0008 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 15:02:24 spark FINISHED 27 s
app-20251217113600-0007 DataLakeHouseTransfer 30 3.0 GiB 2025/12/17 11:36:00 spark FINISHED 8.0 min
app-20251217111835-0006 ProjectSubmissionMediaDetailsIngestion 26 3.0 GiB 2025/12/17 11:18:35 spark FINISHED 21 min
app-20251217111534-0005 DataLakeHouseTransfer 10 3.0 GiB 2025/12/17 11:15:34 spark FINISHED 2.4 min
app-20251217110315-0004 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:03:15 spark FINISHED 3.7 min
app-20251217110259-0003 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:59 spark FINISHED 3.2 min
app-20251217110258-0002 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:58 spark FINISHED 2.5 min
app-20251217110257-0001 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:57 spark FINISHED 2.0 min
app-20251217110257-0000 DataLakeHouseTransfer 6 3.0 GiB 2025/12/17 11:02:57 spark FINISHED 1.7 min