Menu

1 / 2

scrape data masjid SIMAS Kemenag

DEV Community·Eko Priyanto·6 months ago
#qeRwfkFv
#automation#database#php#webscraping#echo
Reading 0:00
15s threshold

get all URL:

<?php

/** * ========================================================= * SIMAS KEMENAG SCRAPER * SQLITE + MULTICURL + AUTO RESUME * ========================================================= * * FITUR: * - multicurl batch aman * - sqlite database * - auto resume dari sqlite * - anti duplicate * - anti banned ringan * - sanitize utf8 * - low memory * - stable long scraping * - restart otomatis tinggal jalankan ulang * * OUTPUT: * simas.db * * ========================================================= */

set_time_limit ( 0 ); ini_set ( 'memory_limit' , '-1' );

date_default_timezone_set ( 'Asia/Jakarta' );

// ========================================================= // CONFIG // =========================================================

$startPage = 1 ; $endPage = 16000 ;

$batchSize = 5 ;

$baseUrl = "https://simas.kemenag.go.id/page/profilmasjid/0/0/0/0/0?page=" ;

$dbFile = __DIR__ . '/simas.db' ;

$failedLog = __DIR__ .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.