microdatasus 3.0.0

Downloads

• Reworks fetch_datasus() around an internal registry of supported systems and manifests built from the files actually published by DataSUS. Each relevant FTP directory is listed only once per call, avoiding redundant connections and URLs for files that do not exist.
• Selects a single file for each system, period, state, and file part. Definitive/current data take precedence over preliminary data, and current data take precedence over historical copies. Results retain deterministic period, state, and file-part order.
• Uses curl directly for directory listings and downloads, with per-operation timeouts and up to two retries for transient network failures. Downloads remain sequential, display transfer progress by default, and no longer modify the global options("timeout").
• Downloads through temporary partial files, validates file size and DBC contents, and reliably removes temporary files. With stop_on_error = FALSE, successfully read files are returned after a consolidated warning; with stop_on_error = TRUE, any failure aborts the operation.
• Applies vars before accumulating results and combines files only once, reducing repeated copies. When track_source = TRUE, the source column is retained even when vars is supplied, and an existing source column now produces a clear error.
• Strengthens validation of years, months, states, logical arguments, vars, and fractional timeout values. Monthly systems require months, annual systems ignore them with one warning, and "all" can no longer be combined with individual states.
• Replaces the former global 1996 lower-year restriction with historical limits specific to each information system. Existing public arguments, their order, and their defaults remain unchanged.
• Adds quiet at the end of the fetch_datasus() signature. Transfer progress and status messages, including each file name, are displayed by default; set quiet = TRUE to hide them. Warnings and errors remain visible.
• Uses immediate cli warning alerts when month arguments or state selections are ignored, including when quiet = TRUE.
• Standardizes download messages across microdata and auxiliary tables, with consistent status verbs, semantic cli formatting, indexed file progress, and one diagnostic item per failed directory or file.
• Hardens fetch_cadger() and fetch_sigtab() with shared timeout, retry, and temporary-file cleanup logic, and adds ZIP integrity and schema validation.

DBC support

• Removes the dependency on read.dbc and reads DBC files with vendored C code adapted from healthbR. Thanks to Sidney Bissoli for the open-source code.
• Adds read_dbc() for local DBC files, with optional preservation of the column types inferred from DBF metadata.
• Hardens read_dbc() by rejecting directories and empty or unreadable input files before decompression, validating missing, unknown-size, and empty decompression output, and wrapping DBF read failures in contextual errors with stable microdatasus_dbc_error subclasses.
• Expands local, CRAN-safe DBC tests for argument validation, type preservation, leading zeros, paths with spaces, repeat reads, temporary-file cleanup, and specific malformed header and compressed-stream errors.

Processing, documentation, and testing

• Adopts dplyr::recode_values() because dplyr::case_match() is deprecated.
• Expands and corrects the documentation for all public functions, including network behavior, actual return types, compatibility arguments, and links to the corresponding chapters of the book Sistemas de Informação em Saúde no Brasil.
• Replaces skipped download tests with deterministic simulated listings and transfers. The suite now covers supported systems, historical, current, and preliminary files, multipart data, complete argument validation, retries, timeouts, empty listings and manifests, cleanup, corrupt and empty files, partial success, vars, track_source, multiple states, and auxiliary-table schema validation. Local file:// fixtures exercise curl without network access; live DataSUS smoke tests remain opt-in and are always skipped on CRAN.
• Runs the active GitHub Actions R CMD check workflow for pushes and pull requests involving the dev branch as well as main and master.
• Requires R 4.1.0 or later.

microdatasus 2.5.0

• Function process_sinan_malaria, fix variable type when pre-processing the variable NU_IDADE_N (#132)
• Function process_sinasc, fix typo when pre-processing the variable PESO (#141).
• Function process_sia, fix typo when pre-processing the variable PA_MUNPCN (#137).
• Function process_cnes for CNES-ST, fix MICR_REG and DISTRSAN conversion.
• Function process_sih, fix NACIONAL and HOMONIMO codes, add more FAEC_TP codes (#131).
• Remove dtplyr package dependence.

microdatasus 2.4.3

• Remove dtplyr usage due incompatibility on recent version.

microdatasus 2.4.2

• Fix error when processing data with process_sinan_chikungunya().

microdatasus 2.4.1

• Fix typo in process functions that was changing variable names (process_sim(), process_sinasc(),process_sinan_zika(), process_sinan_malaria(), process_sinan_leishmaniose_visceral(), process_sinan_leishmaniose_tegumentar(), process_sinan_dengue(), process_sinan_chikungunya(), process_sinan_chagas()).

microdatasus 2.4.0

• Update documentation on fetch_datasus().
• Fetch SINAN-CHAGAS data.
• Process SINAN-CHAGAS data.
• Fetch SINAN-LEISHMANIOSE-VISCERAL and SINAN-LEISHMANIOSE-TEGUMENTAR data.
• Process SINAN-LEISHMANIOSE-VISCERAL and SINAN-LEISHMANIOSE-TEGUMENTAR data.
• Fetch SINAN-LEPTOSPIROSE data.
• Fix message when downloading data for all UFs.

microdatasus 2.3.5

• Treat age field NU_IDADE on pre-processing SINAN on old files.

microdatasus 2.3.4

• Fix error on tabNaturalidade, issue #123. Thanks @SophiaDamianoRovere and @hafermoraes for the suggestions.
• Fix FANTASIA names on CNES, issue #125.

microdatasus 2.3.3

• Add leading zeros to CNES TIPO_UNID processing.

microdatasus 2.3.2

• Fix processing of ESPEC variable on process_sih function (#113).
• Use read.dbc package from Github repo.
• Update TP_UNID processing on process_cnes function (PR #112).

microdatasus 2.3.1

• Import read.dbc package from CRAN.
• Remove CADGER data.
• General package adjustments for CRAN submission.
• Examples of process functions use data samples.
• Update package citation.

microdatasus 2.3.0

• All process functions uses dplyr verbs and dtplyr for fast processing.
• All process functions returns a tibble.
• Add track_source argument to fetch_datasus function to create a variable called source with the file name that originated the row.
• Updated CBO (CNES), CADGER (CNES), EQUIPE (SIA), SIGTAB (SIA) reference tables.
• Sample data for health information systems.
• Data files with XZ compression.
• Internal tests restructured.
• Progress bar removed from process_sim due increased processing time.

microdatasus 2.2.7

• Enhance messages of fetch_datasus.
• Timeout the connection check when DataSUS FTP is down or not reachable.
• If there is no Internet or the DataSUS FTP is down or not reachable, the fetch_datasus will return NULL.
• process_sim shows a progress bar.

microdatasus 2.2.6

• Age correction for SINAN data.

microdatasus 2.2.5

• Patch to correct Invalid multibyte string error on chikungunya processing.

microdatasus 2.2.4

• process_sia and process_sih internally uses now data.table and dtplyr

microdatasus 2.2.3

• As the package {read.dbc} is not available on CRAN, this patch uses the Github version.
• Issue #89 points out that files from SIM-EXT older than 2006 present 7-digits variable lengths for CODMUNRES and CODMUNOCOR. Those codes are now truncated to 6-digits standard by process_sim function.
• Related to #66, #84 and #86. Some files are very big to download, especially those from SIA and SIH. A timeout argument was added to the fetch_datasus with a default of 240 seconds.
• process_sinasc pull request #91 fixes issue #90, related to CODOCUPMAE variable
• Related to #79, now the function process_sia downloads an updated version from the SIGTAB table from DataSUS when nome_proced is TRUE.

microdatasus 2.2.2

• process_sinasc correct old code for missing on ESCMAE

microdatasus 2.2.1

• process_sinasc with new codes to process sex variable, avoiding missing results.
• process_cnes corrections to avoid NA introduction by coercion.
• process_sih corrects MUNIC_RES type

microdatasus 2.2.0

• fetch_datasus function uses read.dbc function with as.is set to TRUE for better performance.
• Minor bugs corrections.

microdatasus 2.1.3

• process_cnes uses {data.table} backend for performance

microdatasus 2.1.2

• read.dbc is back on CRAN

microdatasus 2.1.1

• Patch to use development version of read.dbc package.

microdatasus 2.1.0

• Bug correction that impacted download of all UFs from monthly data health systems.

microdatasus 2.0.6

• Updated codes of tabNaturalidade
• SIA-PA fetch files bug corrected.

microdatasus 2.0.5

• Bug correction to download CNES-LT files.

microdatasus 2.0.4

• Bug correction to download all UFs.

microdatasus 2.0.3

• SINAN-Dengue bug correction.

microdatasus 2.0.2

• SINASC bug correction (ESCMAE2010).

microdatasus 2.0.1

• Ages in minute unit.

microdatasus 2.0.0

• Integrated download of old, current, and preliminar data.

microdatasus 1.4.8

• Tests correction
• Typo correcntion

microdatasus 1.4.7

• Bug correction for download with newer R versions.

microdatasus 1.4.6

• Bug correction at process_sia function.

microdatasus 1.4.5

• Bug correction at process_sim function.

microdatasus 1.4.4

• Updates SINAN functions for DataSUS changes in file structures, from per UF basis to all data (BR).

microdatasus 1.4.3

• Correct SIH COD_IDADE value 5 for ages above 100 years.

microdatasus 1.4.2

• Correct codmun handling for CNEST-ST data. Issue #38

microdatasus 1.4.1

• Check local Internet connection and DataSUS FTP server availability before download.
• Argument to fetch_datasus to stop download if there is an error.
• Now is possible to download preliminar data from SIM-DO and SIM-DOFET with “SIM-DO-PRELIM” as information system at fetch_datasus.

microdatasus 1.4.0

• Download and preprocess SINAN Malaria files.

microdatasus 1.3.1

• Update functions documentation.

microdatasus 1.3.0

• Download and preprocess SINAN Zika files.
• Minor error message corrections.

microdatasus 1.2.0

• Download and preprocess SINAN Chikungunya files.

microdatasus 1.1.4

• Minor correctiont at process_sinan_dengue function.

microdatasus 1.1.3

• Documentation correction process_sinan_dengue function.

microdatasus 1.1.2

• Document and export process_sinan_dengue function.

microdatasus 1.1.1

• Fix NAT_JUR field at CNES files.

microdatasus 1.1.0

• SINAN DENGUE files download and processing.

microdatasus 1.0.0

• Complete overhaul of the package, meeting current R package standards.
• All functions revised.
• SIH code tables updated for COVID-19.
• Process CNES ST and PF.
• Added a NEWS.md file to track changes to the package.