How do I fill my job board from an XML feed or a careers page?

Indeed publishes its XML feed format: a source element holding one job element per job. Indeed requires children such as title, date, referencenumber, requisitionid, url, company, city, state, country, email and description, asks for several more when the job is on the career site, such as jobtype, salary, education, experience and expirationdate, and takes optional ones such as remotetype and category, of which it strongly recommends category. Leave Another feed format empty, and JobClay reads that shape as it is:

A feed in another shape takes a mapping, typed under Another feed format. Name the Job element, the one that holds a job, then for each field you want the element inside it: a child by its name, a deeper one as a path such as pay/min, or an attribute after @, such as pay/@currency. Title and either Job id or Job page address are required.

Press Preview before you save. It reads the feed with your mapping and shows the first job as JobClay reads it, with every element found inside it. Pay is kept when its currency is your board’s and its period a year or a month.

A feed is parsed as a stream, one job at a time, with entity resolution, DTD loading and network access turned off, so a feed cannot make the reader open a file, reach another host or swell in memory. The fetch reaches public addresses only, on http port 80 or https port 443, checks each redirect again, and stops at a time limit and a size cap.

A run that meets a broken feed keeps the jobs read before the break, and one that meets a feed longer than a run reads keeps its first jobs; neither closes anything.

A company that wants its jobs in Google’s job search puts JobPosting structured data on its job pages. Google asks for it on each job’s own page, not on a page listing many jobs, so JobClay reads both levels: the careers page you give, and each page of the same site it links to, one level deep. It reads the JSON-LD form, a script of type application/ld+json, which Google recommends; microdata and RDFa are not read. From each posting:

Before reading any page of a careers site, JobClay reads its robots.txt and follows the rules for JobClayBot as RFC 9309 sets them out, the most specific matching rule winning. A missing robots.txt lets it in; one that answers a server error, asks it to slow down or does not answer keeps it off the site for now. If it refuses the careers page, the run brings nothing in and its log says so; a refused linked page is skipped.

A run’s requests to a site go one at a time with a pause between them, longer when robots.txt sets a longer Crawl-delay, each naming JobClayBot and the page that explains it. A careers source runs on a daily or weekly schedule, never hourly, and Run now starts it at most once every 60 minutes. A run reads only so many linked pages for only so long; stopped early, it keeps what it found and closes nothing.

A page with no markup gets a line in the run’s log, which names the company’s board on a hiring system JobClay reads if the page links to one.

A rule condition on a source field reads one element of a feed’s job, or one text property of a JobPosting, by its name, whatever its case. That is how jobs from feeds and careers pages get sorted by what their source already says about them, and, since JobClay reads no experience level from either kind, how they get marked entry level.